{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个实例中，我们使用tensorflow来实现一个简单的手写数字识别的网络，并用这个网络来做个\n",
    "简单的识别示例。\n",
    "\n",
    "首先导入一些用到的库。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:50.515650Z",
     "start_time": "2018-06-01T06:32:43.133728Z"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "from matplotlib import pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "tf.logging.set_verbosity(tf.logging.INFO)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T05:39:48.304594Z",
     "start_time": "2018-06-01T04:55:17.674707Z"
    }
   },
   "source": [
    "先来看看数据长什么样子"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.075054Z",
     "start_time": "2018-06-01T06:32:50.518290Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting ./train-images-idx3-ubyte.gz\n",
      "Extracting ./train-labels-idx1-ubyte.gz\n",
      "Extracting ./t10k-images-idx3-ubyte.gz\n",
      "Extracting ./t10k-labels-idx1-ubyte.gz\n",
      "(55000, 784)\n",
      "(55000, 10)\n",
      "(5000, 784)\n",
      "(5000, 10)\n",
      "(10000, 784)\n",
      "(10000, 10)\n"
     ]
    }
   ],
   "source": [
    "mnist = input_data.read_data_sets(\"./\", one_hot=True)\n",
    "\n",
    "print(mnist.train.images.shape)\n",
    "print(mnist.train.labels.shape)\n",
    "\n",
    "print(mnist.validation.images.shape)\n",
    "print(mnist.validation.labels.shape)\n",
    "\n",
    "print(mnist.test.images.shape)\n",
    "print(mnist.test.labels.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T05:49:40.128071Z",
     "start_time": "2018-06-01T05:49:40.123888Z"
    }
   },
   "source": [
    "可以看到images里面有数量不等的图片，每张图片是28x28长度的一个一维向量，\n",
    "所以用的时候需要先给它还原成28x28的二维图片。labels中则是图片对应的数字的值。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.695746Z",
     "start_time": "2018-06-01T06:32:51.077167Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAAHiCAYAAADf3nSgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzt3Xd8lEX+wPHvpBBKQleUXkNTyin2ggIWxN4b6qEo2DgV9fT8eZyn3lkBsVFU7PVQsWEFGwoiSpFepIP0HpLs/P5ImHlmzYbNZjeTbD7v18vXfWdndneOJ7vffWaemUdprQUAAJS9FN8dAACgsiIJAwDgCUkYAABPSMIAAHhCEgYAwBOSMAAAnpCEAQDwhCQcoJTSSqkdSqn7o2zfTym1vfB5rRPdP5QMxzO5xHA8hxS210qptET3DyXHZ1REsVmHpZTSItJGa72wsHysiHwc1qyGiJyntX4n0vNQPhRxPOuLyHsi0k5EUkVkjojcprX+rrjnoXwo7rgopa4QkRdE5Bqt9ejA481FZImIpGut88qmp4hWUcdUKZUqIkNE5K8ikiUiC0XkBK315uKeV1Hx67AYWutvRCRzb1kp1V1ExovIJ776hFLZLgUf7AUiokXkTBEZr5Tany/oikspVUdE/i4is333BXExRESOEpEjRWSZiHQUkd1ee5RADEeXzBUi8rbWeofvjqDktNa7tdbztNYhEVEiki8idUSkrt+eoZQeFJHhIrLed0dQOoU/qAZJwYjG77rALK01SbiyU0pVF5HzRGSs776gdJRSM6Tgl/X7IjJaa73Oc5cQI6XUYSJyqIg847sviIuDRSRPRM5TSq1RSs1XSl3vu1OJxHB09M6Vgl/ak3x3BKWjte6klKoqImeLSBXf/UFsCucOnxKRG7XWIaWU7y6h9BqLSC0RyRaRFiLSRkS+UErN11p/5rVnCcKZcPSuEJEXNVeyJYXCoenXROROpVRn3/1BTAaKyAyt9WTfHUHc7Cr8339prXdprWeIyOsi0ttjnxKKJBwFpVQTEekuIi967griL11EWvruBGLSQ0TOLhy2XCMFF/M8qpQa4blfiN2Mwv+tNCc7DEdH53IR+V5rvch3RxA7pdQRUvA3P0UKlijdJCINRORHn/1CzK4UkaqB8v9E5G0RGeOlNyg1rfUipdQ3InK3UuomKfiBfKGIXOy3Z4lDEo5OXxF52HcnUGoZUnAVbUsRyRWRmSJymtZ6lddeISbBdaMiIkqpPSKyVWu9xVOXEB8XS8EPqQ0isk5E7tFaf+G3S4nDcLQrR0SmKaXuCz6otW6ntf7Tr2ul1FVKqc2FzwuVUR8RPed4aq0naa07a62ztNZ1tdbHa62/3tuY41nuFfn53Etr3T1so457ReTXwudVmuHNCuZPx1RrvVJrfYrWOlNr3VJr/ezeumT8jLJjFgAAnnAmDACAJyRhAAA8IQkDAOBJmV4d3SvlfCagPfks9FbctxPiePqTiOMpwjH1ic9ocon2eHImDACAJyRhAAA8IQkDAOAJSRgAAE9IwgAAeEISBgDAE5IwAACekIQBAPCEWxkCAMq/lFQTzh/V1amaffJTJj79igEmTvtiWuL7VUqcCQMA4AlJGAAAT0jCAAB4wpwwAKDcSWvWxCnPf7CeiZd0Hx3WuoqJNreycf0vEtK1uOJMGAAAT0jCAAB4wnA0Ko3UDtkmnjugjlO34JynTRwS9xasKWJvC/rU5hYmHvtYb6ddvTGT49JPoLJKa9ncxL/dXd+p+/MQtHXN8qNN3OCb9SbOj1/XEoYzYQAAPCEJAwDgCcPRSCppTRo75d/uPcDEr534rIm7ZoScdqHA79GQuHXB36r9ay80ccM7XnFaPTfhWBPnrVgZfafxJylVq5q46dfKqXuq0XcmTlX22MzZs9Npd+vJfU2cP2+hoHxS6fZq5jn/rGviJT0jDz+3/PyvTrlt/99MHNq9II69SzzOhAEA8IQkDACAJyRhAAA8qfRzwqtvOcopq8DqlKobbGFTO/d5B062F79XHT8lIX1DdBY/dKSJ5176pFMXXG4UXGoUCvv9+eHOWiaesr1lxPc6pMZSE5+budWpWzVhlok/6OgugcK+BeeBV75ul4J90OiVopqLiEj3WWeZWD3qLmnJWPRLqfuU1rypifOWLiv16+HP5o3obOIlPUdFbNd64pUmbtP3Z6cu/CqOioQzYQAAPCEJAwDgSbkcjl53vTtEvLlTronHnTQiru/VvsrUiHW7dZ6Ja6VUc+rWXb7DxKuGu/+Mj63pZeINF9Q0cd7yFTH3E5Gd38suWQnf7cpdbmR/cz65uZXT7rOTO5q4uOVF351+kYnPeOZppy64fOkD6VZ8p/EnC/9pb9Q+t9uTEdu1+eJqE7cdMM/EoR1LnXbuX0J05o90j9t7Jz1h4gtfuMWpa/rP72N4BywceoRb7vNUoGQ/oy0/c5chZfefbeJYjm15xZkwAACekIQBAPCEJAwAgCflZk54/ig7FzO39zCnLkOlB0tl1KPw93Xtn1ojELt1Lzb72sSXvdHdxJsuaeq0Y8lDKRx2sAmvq2fnZj/ceYDTLLjcaNbWhibOGbyf027RQ/YgZt9X3anLn2O3wQsuR0t/1j3wuYGJqpV3uNc1NPov84fh9JGdnfLXlzwcKNljsCzP3Y4yu59dChbK3VPqfuT2PMTE43q515x0DGypiNjtOcV+v487a6hTl6rs0jRnGdJVvzrtdKgi3BOp5DgTBgDAE5IwAACelJvh6KdPeNHE4cPA/93QxsTr9mTF9Pr/m2aHnJqOV8W0jM6KHvb3y0O9X3Xqgjspvdx8ookve7W7027ThfaOPyxfKqEpM03Y/9wBJk5dvdFp5i43WmOilXe4u2LNOd4uRTl11DVOXeocG2/oZ3fnytXTnHbB5VDNXvnd7Ud4/yFr73CHkvdPtUPQu7St6zvoVqdd9dwf49qP7X+zn9eDq7jfPdt1jolbvLXBqUvOwdHEqHf3EhN3qlLVqes153QTZ99rj0V+kg4/h+NMGAAAT0jCAAB4Um6Go4deeJ6J/9GlplO3/7t2V5z8De5wY7SyJfLOWLFoPd7Go5/r7dSted3u4HR97eUmDg5Ni4i07W+HUZvfw3B0rPRUOzQd7bBv1fXunjsjtzQ3cZW12526xUPslc4vXG6HrYM3hBARmZZjf9MWt+sWCvTP/jZi3dnzzjdx9XGRh59Vmv0KU9WqRWwXLv9gOx3xePvnI7brPu0qE+8/e27Urw/XTY0+j1i3daydlqu9YHJZdKdc4UwYAABPSMIAAHhCEgYAwJNyMyesp9k7ZNRzV36U+6UAoRnuXNHzj/cx8fVDng5vbrx6md0Z7K57Dot/xyqhXWe6/44b29k/8eA8cL2Z7rxv/1pLTdzlA3d50WEZ9nnBZUhTc9zfsP/oZ5c2pYp703GUTFb6bhPvCKvLPelQE9e9Z6mJ32j5aQneYVKRj34Xdkz3+0/Z7dCXbLZcZu+WdFzVX0x89IxznHa1X/qhzPpUHnEmDACAJyRhAAA8KTfD0UA8rLrQ3YVpzvF2OiC4pCgUdlvwYF1w+Dm8LrgM6fK3b3Datfyq8i2vKI2Ro053ytfdZm+e8GJLuwbwuu9PcdqNaWaPaZqE3T2llK4cf51TbjO5cg+VlsbmM8InEgrsfN+9yUqmXpy4TqSE/X2Uw124OBMGAMATkjAAAJ4wHB0HK+5y7x0b6rotquc1SLVDp3knHuLUpX05Lbw5YhC8mjn4m9N9vPi6/stPNPHyv9ubiTD8XDo7GocfA6uasvfxHdvsy7BaO8R46xp7NfxHE7o5rXIPtJ+vhSeNiqpP9X8u/c1dUODAOluLfLzahsjHPVY5p9pjv/4ae//pgxqsdtptO8/+XeWtXiPlAWfCAAB4QhIGAMATkjAAAJ5U+jnhtJbNnfLCfgea+KmLRkb1Gt2rursjparofts0Tss08cjnhzl1A5sdE9VrwNXwjSpO+fxGdhnMQTVXmfi6et877RoFbigf/tt00YPtTVztqylx6CVERLKf/cMpt8+9PqrntX7J3kktNG+RiVvkuXP0i/9zZFSvN3Dl0Sau+6p7LYYOb4yI0g5o4JRHtX0lUMqU0kqtXcvEZ01e4NRdmDXcxLVSIt9Nq+OIS03c+FzmhAEAqNRIwgAAeFJphqO3n3+4if/4i/3t8a9zXnfaXZS1KYZXL/1vmZ6fD3LK2fJTqV+zMqr2njtcnPOejacFjlP/bgOcdtvus7v7fHnwG07dMf+0uyb9Oq2JifNWrCxVXyu7/PmLnHKLOxdFaBn2vChfP21ndMuNfhrdxcT1c1l2FrP0dKfYNK10Q9DrBrpLP8+6dqKJ+9daFdY68hB00H5ZRe/i5RNnwgAAeEISBgDAE5IwAACeJNWcsOra0cS1R7jblX3U3N55JdolRO/usHMas3Y1jtjug4e6O+XUHLuw4Yp/2bvB/Hkew6qyJj1iXWWR1sT9N85bviJh76WnznTKmYEb9Zw/yb27z7jWH5n4oKvt0rGm/2ROuDxTxUwe5wVmluvMzymD3iQ/vc3drnfkloYmLu67L7V+PRMv/2tbE88c9FQce1dgy66qJt4/7q8eG86EAQDwhCQMAIAnFXo4+vch7iXs91xkl5ZcmrXBqVuWZ++sMXdPHRPf+NrVTrvqq+2yhgMnrjdx/m/zI/ajlkS+8feCvwd2kQkbklmSu93Ezd/bLpXRrjPtXXCCS4FERD743U4vHHjWnDLr05ZHmjrl0DN2eiG3za4y6wdK56qLJ0SsO3+hnXJInfhzxHaIXv7mLU75tRX2zkb9a9m1gkff8aPTrtt9i018QeYXce3TkD86OOWGN9klSnlxfafYcSYMAIAnJGEAADyp0MPRtbutc8rBIegev53h1OU+cYCJg7sqNZfIO+REuzNPuNDxXU18Vu0xgRr3N8/GUOBmA1Pcq3WTWfAq6Asf/NjEP21t7rQryyHo4Obw5/3HHcZMEW70XhGk7refU26TsTBi2/VPNzdxlpSPjfyTze7n7c1wch7ONfHDB0yP+3vlavtt3WFSPxNn/92dlsz7fXnc37u0OBMGAMATkjAAAJ6QhAEA8KRCzwnX6+cu62l9i70zTqvB7lxvmiwrkz6JiGzKtruyHF018u+c/rMuM3F9ibwEKtn8foldAhRcuvD49J5Ou1YS/7kj47CDneKpz39t+1TbnUsMBX6rps+P7m4tKHtbTmjllE+vbuf2t2t3V6yq63MFiVXzVbvk8Md/2x0Bj6taVOt9y9chEx/60yVOXZW37bLTli/Z7/7ysgypOJwJAwDgCUkYAABPKvRwdN5qd2lBq8HlY6nBhm5FD4LM2bPTKWc9VavIdsmu0Vd2o/f0m1NNfHOXL512Y248zcT1ZrvDiWlfTivytVM7ZDvlVT3qmzjzNPv38dXBLzjtgsuQQmG/TbM/vtbGQ74v8n3h3xVD3o9YtyTXPabpnxf994Oy1+7by02sZmWZuMXw2U47nW+Ho/ffNjfxHSsjnAkDAOAJSRgAAE9IwgAAeFKh54TLi5NnbXXK42o/GSjZrSmvmH2F067Ox1MT2a3yK7BF59EzzjHxlwe/4TS77s4nTBySkFM3ZN0hRb70GbVec8pdM+zzUgK/OcNfL/h7tO3b1zs1HR62W91VhCUPlVW91Mh3Intk9clhj2xObGcQUYenBzrl5g/abYR1nv2ExbptcEXDmTAAAJ6QhAEA8ITh6Dg4r+YMp1w9JdPE83PtTaSrj6hdZn2qKGpfs8fEQ953h5gfaGD/XXO1+7z79v/FxCGxleF3PAouN1qbv8vET204ymn36YijTdxmjLvbGkPQFd+eUOq+GyFh7m/ZxcRNxF3mp8MbVzKcCQMA4AlJGAAATxiOjtG6gXY4s0Gqe5Xzklx7lebFDww2cf2P3WFOiOQtX2HiX09v4tS1/m/RV0CLiMzpPtrEx824wMR/bKwZ8Tmth9qBZT11plNXTzg2yWxU8w+c8iGP/s3ErW79Ibw5UGY4EwYAwBOSMAAAnpCEAQDwhDnhKKmMDKd87nX2jj/bQnucut5TBpi46bPMNUYrb8VKp9zq0pURWor0ETtfXFMWBeLIKvtSiGR39+uXOuV2fR+zcbr7+ZWQu5QN8IUzYQAAPCEJAwDgCcPR0Qq5g5kvjT/BxB//2t2pa/omSx6Astbs/9ypn1v+78iIbVuxJA3lBGfCAAB4QhIGAMATkjAAAJ4wJxwlnesuQ2p+N3NKAIDS4UwYAABPSMIAAHiitGYfIQAAfOBMGAAAT0jCAAB4QhIGAMATkjAAAJ6QhAOUUloptUMpdX+U7fsppbYXPq91ovuHkonhePYsPJ4hpVTPRPcPJcPnM/nEcEyHFLbXSqmk2OeCJPxnnbXWd+8tKKVGKqXmFX4xXxlsqLUeo7XOLPMeoiTCj+eJSqmflVJblVKLlVL999ZprT8vPJ7LvPQU0eDzmXzCj2kXpdQ0pdTOwv/tsrdOa32viHT00ssEIQnv268iMlBEfvbdEZSOUipdRMaJyLMiUktELhSRx5RSnb12DKXB5zOJKKWqiMh7IvKyiNQRkbEi8l7h40mJJLwPWusntdZfiMhu331BqdUVkZoi8pIuMFVE5ohIB7/dQqz4fCad7lKwnfJQrXWO1nq4iCgROdFrrxKIJIxKQ2u9VkReE5GrlFKpSqkjRaSZiHzrt2cACnUUkRna3UVqhiTZEHRQUkxsAyXwmoiMFpFhheUBWuvlHvsDwMoUkS1hj20RkSwPfSkTnAmj0lBKtRORN0Skr4hUkYJf17crpU7z2jEAe22XgimjoJoiss1DX8oESRiVyUEiMk9rPUFrHdJazxORD0XkVM/9AlBgtoh0UkqpwGOdCh9PSiThfVBKVVFKVZWCiwPSlVJVlVL8u1VM00WkTeEyJaWUaiUifaTgCltUQHw+k85EEckXkZuUUhlKqRsKH//SX5cSiz/WfftURHaJyFEiMrIwPs5rjxATrfUiEfmriAwXka0iMklE3hGRMT77hVLh85lEtNZ7ROQsKZgy2iwFn9ezCh9PSiRhV46ITFNK3bf3Aa11d621CvtvooiIUuoqpdTmwueF/HQZxSjqeL6ptT5Ia52ltW6stb5Dax0SEVFK9Sg8ng2k4Nc4yhc+n8mnqGM6XWt9iNa6mtb6L1rr6XvrlFL3SsHIVY6IJMV9eLmfMAAAnnAmDACAJyRhAAA8KdPNOnqlnM/Ytyefhd5S+25VMhxPfxJxPEU4pj7xGU0u0R5PzoQBAPCEJAwAgCckYQAAPCEJAwDgCUkYAABPSMIAAHhCEgYAwBOSMAAAnpCEAQDwhCQMAIAnJGEAADwhCQMA4AlJGAAAT8r0LkoV2ZLXOznlb49+2sSX9L3RqUv96ucy6ROAyBY9eoSJbz7lY6fuo4uPNHFoxtwy6xP24Qj7PbvkZvcmRPOPH2vi1hOvdOpaXfJLQruVSJwJAwDgCUkYAABPGI6Okl5WwynXO7aaiTe2zXDq9vuqTLqEOMo5rZuJN16z3amb3u2VqF7juhXHmvjbjzs7dS2fXWzivNVrYuki9iGtUUOnPOLM503cq9oup27s4b1NXG9GYvuF4q0ZdJSJH7jhOROfVG2H0y5X23jYYa87dcOlXZGvvfbGo5xyw1ft1EP+ho0l7msicCYMAIAnJGEAADxhODpKNVaoiHUHXPi7U85/JtG9QSxUehUTz3+sq1P34emPm7h1uju9EIry9Z9p/I19zjVfO3VdDu5r4sbnMhydCIuubeaUw4eg4Y/KsJ+pTRf8xan7+rZHTVxdVZHSWvF3OwQ99fqhTt2b1zc28fCh5zp1+z0zudTvHQvOhAEA8IQkDACAJyRhAAA8YU44DnblpTvl0s9qIBHmPdHFxPNPf8qpS5GqJg6Jlmj0X97dKY9uMili2+Fd7JKKR+sdb+LyskwiGTQ5eoXvLiCCxf+088Cz+44Iq43uG/OZzS1N/OxLpzl1jeR7E+fUs1dxpKtUp92lWatN3O3Ox5y6y+UWE5fl/DBnwgAAeEISBgDAE4ajo1TztNUR67a84+7Us5/8HqElEi24DEnEHYKe3Sc4DOYOU63O32ni48bd5tS1HLfHxBkL7PKi/PUbnHZd37jUxNO6vezU/byruYn1ntwIvUdJ7e5zmImHtXwirDZd4E9wWVKNDptK/PyPd2Y55XduP8nEjT78Prx5iWWHfVe8/vdHTHxy10G23bVTS/1exeFMGAAAT0jCAAB4QhIGAMAT5oSLkd/dXlY/vuOTTt0ve+ycYoNXZjl10W5ziPhbff2hTnn+6cF5QnvMxmxp6rT73zW9TNzmux8ivn5eMe+dkxN5DnL8Snuz8mrblhTzKiiJXfXsMT24CnPAPqk0N50s+pf9/vzt0PBlSUULLvtbd647J5yxMrq52eYf2ms4OjW70qmbduQYE4cvX2qRZpcp1pxbdn9LnAkDAOAJSRgAAE8Yji5Gfob9jZKp3Dvr5Gq7q1Jo27Yy6xOKN6D/e045Rezdrx7c0MHEk8/Idtqppb9E9fqpNWuaeMXVBzl1t3f6n4mn73EnJaqdzBC0T9/luOcbWcuLm1hALHJ6uncm++2y6Iagb151tInXnmaHgfM3rIqpH6lf/Wzipl+5dePmHWjiCzLXxfT68caZMAAAnpCEAQDwhOHoYiw9m98oFU1+2O/K4M0YPnqgu4mzlka+AlpS3Ksm84/vbOI+I74w8XW13bGu4ND3afPOCnvRlZHfDzFrd93sqNoNXdHLKVf5JLG7IFUWa286ysQDB7wb1XOCw88iIkuOt5/Z0M7Kd0MTsgwAAJ6QhAEA8IQkDACAJ8wJFyPrAJYeJZPqa/bsu5G4c8AiIh+/PCqq5529sLeJU87d6dTlR/UKKKmBDYLz8ipiu3kft3HKjeWPBPUouaV0bu+U/3OT3YGqR7Wd4c2N4E5YwWVIIomdB1ZdOzrl5uk/R2gpsjA3x8S1FpfdEjbOhAEA8IQkDACAJwxHI6ks2NXAfaDWUhM+9+JwE/9nbU+n2cTfW5v4k8OGi6uaibaEdpu424d/c1q1u9Uulwnt2BFtl1EGmr3rDj8zPRCbY19yh3OLG4IOmvruwSZutOH7uPapOPMGVHfKh2XoCC1FJuywO+pVe29KwvoUjjNhAAA8IQkDAOAJw9FhUqrae0oe0yjypvuj1h0fKG1PYI9QEnOu7+A+8M6PJjww1Q4rD2v4ndMspaEdIgsFhp/DnfDEYBNnP+QOq3Ef6bIR3KWpbXrwGFR12q3MDwyV5jEAHav11x5p4gF1Hg2rtTe2WZ2/y6m55Xe7a1zT/601caKPRFqLZiaedMrjYbWRP9vfbmwdKK2Pb6eKwZkwAACekIQBAPCEJAwAgCfMCYdJqV3LxE80/Dhiu0nf2hu6t5Ji7siDhMs5rZuJl1/k7nSTUswuSkGpKvB7VLuzuz1mn2Pihg+V3fIKFEhtsL9T7nrJTBPXTKka3tzoPu42E7dZwGc0VtvsFKtkpmREbPfIuhPc5x0bnFctuznWedcfaOLgdSDhNgWWG4qIrBnWysQ1mBMGACD5kYQBAPCE4egwec0b7LuRiDT9JDfBPUFQSqd2TvmAkStNPLrJsyYOibsjTqRlQ3eu6eaU/zflUBM/3WusUzem7csm7nuBHeLMfJMhzjJRv45THN3kkyKbbQ0bXsxawjlGWfrk80OdcguZXHZvruy0k06N7im3rTjVKdd4+8cILROLv1IAADwhCQMA4AlJGAAAT5gTDrP+7t1FPt577hlOucrEX00c+b4cKI31/e12eRPuecSpq+UsTYm8DOnW1UeY+OMv7ZxV9uPulqTZq+1dUx454VKn7uOXR5n4onvtsrUP3nTnKpEY+TWqRNVuZq57x5wDhrKcrCwd+J2/rUG3XHq4iede8GRUz/n+O3eLW19LTTkTBgDAE5IwAACeMBwd5umDXgmU7LXuq7bWdNo1zFtRRj2qPLZddIRTDg5B1wrbGWlOrl0i9viaXiaeN7Sj067Wu7+YuOVuu2TC3VfLlTrpV6fc7s3rTfzr+UNNPO6kG5x26Z/+VMyrIlZZj66Oqt2A6e40QmOZnYjuIIJmd811ymvHx/f10xo3MvGC65s6dT9eFry7U+RdvV7bZpegZj+/yanzNZjOmTAAAJ6QhAEA8KTSD0enNXeHNbKUvaIyVaWXdXcqtfWd3Kucg0PQ43bUdeqev+A0E4d++c3EWWFXOEbaMas4KdXcoe+Of1lq4ozA30QoLbqbQ6Dk0po0NnF25rKI7S5d2tPEza5e5dT5u1a3cjqm9kKn/G4bO72Uv2BxVK+R2r6NiRdcUd+pG3re8yY+qdqOsGdGHoIOGnv9mSZOmz0tquckGmfCAAB4QhIGAMATkjAAAJ5U+jnh3aPdcna6nQ/MD9zcPfNNd4kSEi8lsBPWHV9d4NRl/zI1ru+VWr+eiauPc+d632j5UaDEPHBZWNO7iYnf3/99py5V2XOHTbvtLlkpe9wlJyrd7rSlc/fEu4uVRpvRdonYkN5dnLp797NLAK+qudypS33ffn/O3NlYotGlxiQTX5oV3dK0cO/vsDvZ3fb5RU5dux/ssrVYrhdJBM6EAQDwhCQMAIAnlXI4OjW7lYlvbf5+xHYXL7E7MdV83c8NnyuT+jPcW2FsCu0y8dTeQ526bs8OMnH7//vdxPlr10V8/bRGDU28o3Mjp27QsNdMfFr1LU5dcNjqyc32b6faN3MjtkPiBKeJPmoX+PzOd9u1eXugjW/2szl/MshbvNTEE4Yf49QNGmL/XcN3tetbc6UtBOM42Knd6YUnN9ph8q//2s3E2T9NcdqVx88oZ8IAAHhCEgYAwBOSMAAAnlTKOeE9jWqZuEe1nIjt5r/R1sQNNDcIT7Ss1915u+NaDzbxrwOecOrm93nGxLNPsvdEGrTgwoiv/0p7e4es8Pmr4HKo8HmjW1fb7ffm3mhvBK62/SpIjKob7VFYlLfLqWuVVq3I5+wKmyesvppzjHir+9xkp/x/A3qY+Lr9Jjp17dPju+1v8HqMl4ad6tTVHxns16y4vm+i8VcKAIAnJGEAADyplMPRxbluxbHOhi8NAAAgAElEQVQmbvjaPBNzR5ayV3eu/Vd/ZnNLp65D1RUm7l7VDiV/1vGdYl6xasSaZ7Y0M/HjH/Zx6trcM93EajdD0GUh8y27JPCCAwY7db/8/SkT/3t9OxO/M/JEp12jEUwhJdqibrtNfGfri926Kw8w8cmn/GTiRw90p506vniDiVUxX7StXt1g4vq/TY7csILhTBgAAE9IwgAAeKK01vtuFSe9Us4vuzeD47PQW3G/84DP45nWvKmJF/yndsR2D/7lXRN/v621icdPONxp1+KuijW8lYjjKcJn1Kdk+4xWdtEeT86EAQDwhCQMAIAnJGEAADxhiRIqpLyly0zc4qJlEduNlODSJrsLUwupWHPAAJITZ8IAAHhCEgYAwBOSMAAAnpCEAQDwhCQMAIAnJGEAADwhCQMA4AlJGAAAT0jCAAB4UqZ3UQIAABZnwgAAeEISBgDAE5IwAACekIQDlFJaKbVDKXV/lO37KaW2Fz6vdaL7h5KJ4Xj2LDyeIaVUz0T3DyUTw/EcUtheK6W4Y1w5xHcuSbgonbXWd+8tKKVOV0rNKjzw3yulOuyt01qP0Vpn+ukmohR+PE9USv2slNqqlFqslOq/t05r/Xnh8Yx8b0T4Fn48uyilpimldhb+b5e9dVrre0Wko5deoiTMMVVK1VdKfaeU2qCU2qyUmqyUOnpvw2T8ziUJF0Mp1UZEXhGR60SktoiMF5H3+VVdMSml0kVknIg8KyK1RORCEXlMKdXZa8cQE6VUFRF5T0ReFpE6IjJWRN4rfBwV03YR+auI7CcFx/S/IjI+mb9zScLFO1lEvtFaf6u1zpOCP4hGInK8324hRnVFpKaIvKQLTBWROSLSofinoZzqLiJpIjJUa52jtR4uIkpETvTaK8RMa71baz1Pax2SgmOZLwXJuK7fniUOSbh4qvC/8PJBfrqD0tBarxWR10TkKqVUqlLqSBFpJiLf+u0ZYtRRRGZod7ODGcIQdIWnlJohIrtF5H0RGa21Xue5SwlDEi7eZyJyvFKqe+EQ110iUkVEqvvtFkrhNRH5PxHJEZFvRORurfVyv11CjDJFZEvYY1tEJMtDXxBHWutOUjBqdYkk+Y9kknAxtNZzReQKERkhIqtFpL6I/CYiK3z2C7FRSrUTkTdEpK8U/JjqKCK3K6VO89oxxGq7FHxRB9UUkW0e+oI4Kxyafk1E7kzm6zZIwvugtX5ba32Q1rqeiNwrBcOXUz13C7E5SETmaa0naK1DWut5IvKhiJzquV+IzWwR6aSUCk4ZdSp8HMkjXURa+u5EopCE90EpdUjh/OF+UnBV7fjCM2RUPNNFpE3hMiWllGolIn1E5FfP/UJsJkrBhTs3KaUylFI3FD7+pb8uoTSUUkcopY5RSlVRSlVTSt0hIg1E5EfffUsUkvC+DRORzSIyr/B/r/HbHcRKa71ICpY/DBeRrSIySUTeEZExPvuF2Git94jIWVIwvbBZCo7tWYWPo2LKEJEnRWSDiKwUkd4icprWepXXXiUQd1EKUErtloILdoZrre+Jov1VIvK4iFQVkQ5a68UJ7iJKIIbj2UMKknKGiPTWWn+V4C6iBGI4nveKyC1ScDxraK3zE9xFlBDfuSRhAAC8YTgaAABPSMIAAHhCEgYAwJMy3RS7V8r5TEB78lnoLbXvViXD8fQnEcdThGPqE5/R5BLt8eRMGAAAT0jCAAB4QhIGAMATkjAAAJ6QhAEA8IQkDACAJyRhAAA8IQkDAOAJSRgAAE9IwgAAeEISBgDAE5IwAACelOkNHICyltasiYk3H97IxKv77HHaDfjLJBMPqjPfqTvo26tMHFpaw8Sth/zqtAvt3Bm5HwceYOK81Wv21W0gqeT1OMTEGzpmOHW79rf3mNCtd5j4js6fOu361bKfm092uq8xeGQ/Ezd86PvSdbaMcSYMAIAnJGEAADxhOBpJZdXgo5zy3Ve/ZuKzM9dFfF5K4PdoSEJO3YxjxtjCMTbsvPtmp12zeyMPg2W8kW/ivOMiNsNeyt6Kdd2AI52qATe+a+L+tVbF9PIjtzQ08btnHGHi0NIVTjud605bIHpbLrP/rl/+Z7iJM5SbdkJS9C2PU8S9HW+utu16VHOnfr696VETH5V6q4kbP1j+h6Y5EwYAwBOSMAAAnjAcHSalc3sTz7ulmokv7/Kj0+7GulNM3OPRwU7dAUPL/xBIMkntkG3i4PCzSOQh6D/yc5zy73nVTZwv6U7doVXskGRqYJj016uHOe26bbXD0wc+6v4NHFN3kYknSM0i+1TppaSacPndh5t45nUjIj4lR9th/lV57jGtGhjN3D+1ulPXr6Yddu438W0TD9vU2mn3RZ+DTJy3dFnEfuDPtp613cTpyh7b8OHnZXm7THz3ijMivt6Pc1va16vhThN8e/TTJj7qLLtqYflj7lXUOsf9GykPOBMGAMATkjAAAJ6QhAEA8KRSzgmrDDtPsKb/IU7dj3faeb5tITvvcMTrtzntvu5i546Ov2yqUzdvaFy6iSjNvTPTxOFzwMFjeMJP15i4wbCqTrvUiT9HfP3119olMn0Gfm3iu+r/4rTLd6efHN9ubBUo/RG5YSW2cnC088B5Ju78qp2Hb3n7ZKddavs2Jp779yynbtaJz5g4uGTm5joL3Tf7wIafd2/hVOWv3xCxjxBpfs1KEw/8xK7Lm7XxAKddncBKv/z5iySSbNkYse7wZ/5m4vmn2/nhLrfe6LRr/ED5u16HM2EAADwhCQMA4EmlGY5OqWqHH+cO7WTihae7w15PbLZDWG8NOcXErd4MG+rKtsOLM1p1cer06XZtRNpOu4Qi7YtpJe02ovC/Y58OlNzflQN/t0seGp79W0yvX/9Ze+y/XGe3zLprxC9FNS/SvE/s31VjhqNFRESluV8/VY6Obnj3oP/ZIcY2YUPQQflzFth2fd26Y/vbMdCH7hhp4u5Vc512weHpL7IOdl+E4ehi5W/aZOLpo+yUTu1F7jKh/PmRp4Kilbqj6PPJjr3nOeUtD5T6reKOM2EAADwhCQMA4AlJGAAAT5J2TjilurtN3cpXm5l4YTe7POGxTW2cdhNuPN7EmV/9EPH1g5fSV9+01akbNHmiiUevsZfmb/liH51GTA6uYreZDN8Sb+p8u6wkW0o/h5c1y87nfrvbXeZUb3ZeeHNDq4hVlVZq08ZOeeohrxXZ7onNLZ1yu2fsXGN+eOMo1R9p55LHXXOoibs3jDzHjNjVG+3n37VP/V+d8ivSOEJLfzgTBgDAE5IwAACeJNVwdHAIeu6jBzl1wSHoRza2NfHXZ3Rw2qUuKfnl8suvdIe0e1SbYOKN+9nXe7F2J6dd/uYtJX4v/NkJs8418WcHvenUje0+2sT3i7uULFp5PeyuavvdZ6chWqa5x6/+rUtMvOM99zVU0fctr9SWXtgwYt12bZexvP7AKU5drd8iTxPFYvGVzU383Xj3bmlHZ4RMvKC/29+W99gdoXRe5KkIxF/Oqd2c8pW9JhbZ7t11XcMeKX/LAzkTBgDAE5IwAACeJNVw9B+XdjbxwjOedOo+3Gk3+f/6zI4mzluytNTvu6dW5LHGObvtEBbDz4mROcj+GT/9tjs10L/WfBPPf+owE3f472qn3dqT7FWTp98wyanrW9ve1KNhWvAuDe4dG15sOd7EfXq7G8fnVWM8WkQktV5dE99xxZsR2729zV7VXuuV+A4/h8ufbXdVumJCf6du4Rl2GmtOX/c75bR3Attw/TQrMZ2rxFJr1nTKay+239vXDnLne/rVXGHipXm7TLzhYfemG1UZjgYAAHuRhAEA8IQkDACAJxV6Tjitkbtk4PbBr5p4Zf5Op+7BeweauObi0s8xpbVsbuI+p/4YuSESLni3nJeGnerUDbjX1s09MzCnd6b7GimB36MhCbmVYXO/e92x5kinPP5ru/NSu5krnLprH7J3cJpwjzvXVZmowN3MLs1a57EnRas5N+wr8Yyi24mIzLvO/n/JvjpBHUpCKV3cZaGrutc28da2dqnXNUe712YMrvdVMa9qt6Tr+dEtJs4ePyXGXpYdzoQBAPCEJAwAgCcVejg6VM8d1ju3ht3Y/V/rD3fqar5a8iHo4E3HVw46zKm785o3THxRZvm77L0y2XWmPTbHXjs17q/f7/deJv7jlqYmTpmx0GnXeqf9G2P/pNL5alO7QGmzt34gemkHHuCUr5hkb9pwcvU1Jk4Xd4g4XaWW+r2Puc1ON2a/Ef/vgETiTBgAAE9IwgAAeFKhh6OLc0bN6U75g/43mzh9Z+TdizaeZndb+eCop0zcKs0dQnl3h72ir/X71zl1wV12pm5sFqhZVXynEbWNV9krky+49VMTD6ozP6xldL8zg0NiHZ50d7tqcv/3gZIdGg2/hro4KaokrZPX4qubR9Vu1uv2CtoG8n0xLVFe6Dru9ODZNTYGSlUS+t7ODVJCsd5l2g/OhAEA8IQkDACAJyRhAAA8qdBzwqGZ85xy9pv2MvX5Fzzl1E25170DSjQ+2VXPxGeN/qtT1/ShaSZu13ar+8TALjsLpto54ZbMCccsrVkTp3zPXWNNfGr1bSYO3+1qY769OfwZM+wxfPGgF5x2rdPtrlhpu0vV1SKFNL93RUR2N9vjuwtIlNXuUs3Dp11i4q77rzTxN18e7LSrtlZJUXY1cK/d+de5r5v43Mz1Tl3vuyaa+CPpbuKs1xN7B6544JsBAABPSMIAAHhSoYejRbvDFa3/ZoceDpt7vVMX6r1JirJ5XZZTbv6Ojat8YndeaRK2TCL4znrGXKfu3+sPMvFlJ9tNyL+/PbGX6Seb1LatTfzghJedurbpdknRsjw75Nz75cFOu9ZP/W7iuivt8qU+L7l/H3NPHG3bnRw2bfB4YEefGJc/jHn1FBM3ZskNklD+Jvc7dr8zbDl4O5MWMlli8dITdhfEJ56v7tR9ebDdwXDSNW1sxZthu3GVw+VLnAkDAOAJSRgAAE9IwgAAeFKx54SLUf/ZsHmHZ4tut38c3iu1Xl2n3LW6nZuetrNFHN6hclpwb6aJg3PAIiKf77Jz+f+8/yYTN3/ePe6R7mbU+nJ3W9NzJ51m4gkd33Lqjhhotzzdf0Rs87mNH2AeeF9W5+80cc1l5f8+VDUWco1HWcpbbe/ElHmKW3fr1GNM/FG7d018xDU3OO3+lBfKAc6EAQDwhCQMAIAnSTscXZZ0I3dQ+7Tq20188zf2bj/Z8lOZ9SkZvHDEcxHrHr75chPX/bD0Q0yLPmlpC+4Illw9cLyJ3x9RT5AYWSl2yiGnpo2rJfh9U9vbJS2XXTMh6uc1G7vYxOV/8Dw+UuvUccp6j90BLbRjR1l3x/jk664mfvwiO/Vz9vVfOe2+ebZqmfUpWpwJAwDgCUkYAABPGI6Og5W96kasS1ufXoY9SS6pgX3JUsJ+L2ZsyAlvXirNX7BDiy/3dW8WcXS1hSb+sH62ifPXb4hrHyqDrNmBK4pPdusylb2JxpE3293q5ryY2D41esHukHZLnQUR27Uf6+6y1vKPqRFaJpe0Jo1N3OG9lU7dB+/Z6bamQxK7AkBl2L+PZYMPcepu7/1uePOCPlVZH/ZI4yLb+cSZMAAAnpCEAQDwhCQMAIAnzAnHQU4dve9GKLGXNxxl4q4Nv3Xqlv7Nxi0f7GDi0C+/xfReOs/eXWVLvnuHlvZV7G/VdWfbOeF6o6JfGrXtoiNMXBFuNJ4oTV5fagu3RG53cHV73505ckDc+7H4P3Yu881GjwVqMpx2o7bY6wNaP77QqcvPqxwLk7Yc1sjE/2nwvlN319XfmfiQ+n9z6tqO3lri91p8fm0T59YJOXX39XzbxBdkuvPPKaJMHHzWU/ed57SrJeXvs8eZMAAAnpCEAQDwhOFolFuffv4XW+jrDkfPOGaMiVe9Z5crPbquh9Pu42+6SjTGnTPUxOE3i5ieY3+r7vfKryZ2B8uKd94/PjXxhNdrluCZyUUHdlUatqm1U3dzHTvce3HWMhPf/2Jvp13bR+yNHkIz5kb1vtvPP9wpT7/scRNXCyyNCg4/i4i8f66dEsn/I/LypWRWY+UuE/97/UFO3T/qzzLxvHOecupSzgkOEQeXGyqnXbDOeX6U7URE1gVu/nH0e7eaOPtt90Yt5XHikDNhAAA8IQkDAOAJSRgAAE+YE06AVGV/29SZ7bEjFVzroYtM/OOF7vafh2fkmrhxmr3PzqNhS5kevdAtR5Ii9vVDYbO9H2/rZOt27pRYjJpztImbysyYXiMZ5G/eYuIv+rjzi/KBDYPzwwt6jHaavXSYXbL039fdJShBl57zpY1rPerUVVPVw5uLiMgTL5/plBvPSexWjBXCDzNM+PUtRzpVJ/3dzuv/r90bTl1wG9Lw+d0gd3mRnbV9ZZt7d7rzMu32oh0/GejUNRtnX6PNhz+auDzOAYfjTBgAAE9IwgAAeMJwdALkazucWWfOdo89qdjy164z8X9OOdepmzdwPxP37/GFiQfVjW3HrH7LTjDx1AnuMGnLMcsCpRUSi6bnV94h6Ejyli5zyq8OC9xW6eZAWMfdqeryrDU2vmZElO/mDj+/sLWhid8573gTN57zoyCytC+muQ/Yj56ccfrNTtWqi/eYeMqxdvnSefMuctqt/8De2UgFZoIavuIuPxvb2U4VZH/5U9R9Lu84EwYAwBOSMAAAnjAcnQDBq6MRH/nzFznl1oNs+UupEYi7xfgOdrP5puJeEVs5tun3L3hDjE9fqG/iz5t3cdrNvcFeNXvMYXb64dspHSSSdiM3OeXQ/CUm1rnzSt5Z/EnV8VOccsvxNr5I7M5jaeJOQxwQVt4rP6yc9uXGUvWvvCJbAADgCUkYAABPSMIAAHjCnHACLMq1y5JSN9sdlsLnOAAUTefa5S35CxY7dW1utuW1wceLuWE7nz2UV5wJAwDgCUkYAABPGI6Og+b/mOyUB/7jmEDJXVoDAMBenAkDAOAJSRgAAE9IwgAAeEISBgDAE5IwAACekIQBAPBEaa199wEAgEqJM2EAADwhCQMA4AlJGAAAT0jCAAB4QhIOUEpppdQOpdT9Ubbvp5TaXvi81onuH0qG45lcOJ7JJ4ZjOqSwvVZKJcW9D7g6OkAppUWkjdZ6YeCxkSJyvIi0EZG/aq1fiOZ58C/8uCilskXkYRE5SkRSRWSqiNyktZ5X3PNQPhRxPI8VkY/DmtUQkfO01u9Eeh7KjwjfuV1EZIyItBeROSLST2v9S6C+uYgsEZF0rXVemXY4ATgT3rdfRWSgiPzsuyMotdoi8r6ItBWRBiIyRUTe89ojxExr/Y3WOnPvfyLSR0S2i8gnnruGGCmlqkjBZ/JlEakjImNF5L3Cx5MSSXgftNZPaq2/EJHdvvuC0tFaT9Faj9Fab9Ra54rI4yLSVilVz3ffEBdXiMjbWusdvjuCmHWXglvsDtVa52ith4uIEpETvfYqgUjCqMyOE5E1WusNvjuC0lFKVReR86TgzAkVV0cRmaHdedIZhY8nJZIwKiWlVGMReVJEbvHdF8TFuSKyXkQm+e4ISiVTRLaEPbZFRLI89KVMkIRR6Sil9hORT0XkKa31a777g7i4QkRe1FxpWtFtF5GaYY/VFJFtHvpSJkjCqFSUUnWkIAG/r7WOalkEyjelVBMpmEt80XNXUHqzRaSTUkoFHutU+HhSIgnvg1KqilKqqhRcHJCulKqqlOLfrQJSStUUkQki8p3W+k7f/UHcXC4i32utF/nuCEptoojki8hNSqkMpdQNhY9/6a9LiUUy2bdPRWSXFKwtHVkYH+e1R4jV2SLSTUSuKtzEYe9/TX13DKXSV7ggKylorfeIyFlScEw3i8hfReSswseTEknYlSMi05RS9+19QGvdXWutwv6bKCKilLpKKbW58HkhP11GMZzjqbUeW3j8agTXl2qtl4lwPCuAP30+RUS01u201mPCG3M8K4SivnOna60P0VpX01r/RWs9fW+dUupeKdi7IUdEkmL+nx2zAADwhDNhAAA8IQkDAOBJmd6FolfK+Yx9e/JZ6C2171Ylw/H0JxHHU4Rj6hOf0eQS7fHkTBgAAE9IwgAAeEISBgDAE5IwAACekIQBAPCEJAwAgCckYQAAPCEJAwDgCUkYAABPSMIAAHhCEgYAwBOSMAAAnpCEAQDwpEzvolTRLBj7FxPP6znKqTvxhoEmrj7uxzLrEwBUBqkd2zrlpefUM/GhvWc5dS82+9rEuTo/qtfvcf0Ap1zt3Skl7WJccCYMAIAnJGEAADxhOLo42t6TOSQhp2plDxu3GVdWHcJeaS2amXj52Y1MvC07z2nXNnulice3fd/E2R9c57RrPMH+Hq05fY1Tp7fvNHH+H3+YWKW5H59VNx1m4rxqbn+bPjLNvl5OjgD4s62XHGHi0+6c6NSNqzcz4vNytf38hn9XR/L00GFOefC8vibOn7MgqteIB86EAQDwhCQMAIAnDEfHqFX7VSZWGRlOHcON8bdm0FFO+afBT5g42uGnYKv5fZ5x6/pEfo03th1o4uf+draJVx3rfnxmXuEObwWdPvEaE6vvftlXV4GklVK1qlNe9M+uJp59+QgTR/u5jlV2ehWnPOfmOrbuuvDWicOZMAAAnpCEAQDwhCQMAIAnzAnH6KN275r4zMxeTl0+c8Jxkdq6hYnH3vx4WG3J/3THbd/fxOdmro/6eRdmrbbx6KdMnBL2GzY4gzU9x61L3bK7yHaVzdqb7Nz+1kN3F9MysdIz7FK2Wcc8H7Fdn0aHlEV3kp+yyz2Dc8AiIjMvHx4olf68sMObN0as++2CJyLWPXjCWyZ+/rA+tmJK5KVR8cCZMAAAnpCEAQDwhOFolFuretulQe2rRP69eOLMC01c476aEdulr95s4jENazt1OfXscoWBD73l1J2duW7fnRWRWXu0iQffOtCpqz6Lm3yIiOw4wu4+Nuf4URHbBYf6Y12qEu1rBGte3tokpvfCn4WOtcPOi/vbx387cXgRrf/s7e0HOOV/fGuXBzZ53/0+qPaevflCa/nBxKprR/dFL4j8fsHP+fCWNUycleD7OnAmDACAJyRhAAA8IQkDAOAJc8Iot465fFrEutX5u0y8dmYDE6eeGvn1Gvxk533XHprqvldPuwwh2jngcB9s7WLi6uOYAy5Km4FLTHxO1tlO3ZIrm5o4p46dqVVaYhKqv8fEc3o+G7Fdu4/s/H372xeG1W6K7c0ro8AyJJHweeCRUb3E6fPOMHHonv2cuuzvfoq9b+UYZ8IAAHhCEgYAwBOGo1FuffhTZxM/dPo3Tl3TtEwTz7lkhETlKhumK3c4OlfnB0rub9P1gaHvY9+6zcQTz3/EaXdXfTuk3f2C6526zDd/EIjkb95iC8FYRJrctyKu77X9AnuDeOnp1i3MtTtmtX94o+3fJoafSyJ4R6TwnbCiXYr0Y066ifWJK02sZGVRzZMOZ8IAAHhCEgYAwBOGo1FuZQ+wW9X8pV4/p27m0S+YOJYdlXLDrrh9f4e9ofewJT2cupRh9U3c6iM7rHxsjVucdnNPf9LEq3rlO3XZb5a4iyil1X32RKwbssJu0J8/f1FZdCcp6fatTOzeiCGy9l9c65RbjbSf3xT5JT4dq0A4EwYAwBOSMAAAnpCEAQDwhDlhVAgth7jze907Xh+hZWxq/7TGxNUWLwmrDS/v28HZy51yTiydQqks6DHaxKGw841pU9qYuLVsKLM+JZuVPWqZOKWYc7pxO+qauM2IXLdyykwpK8E+/nmZoo21u/lXQnEmDACAJyRhAAA8YTgaFUL+7HlOOXN2fF8/b99N/qRtduQdfWbOd28Ony1rIrREooREB2J3GVusN4Wo7NKaNHbKp1wy2cTFLRW848sLTZw9ZUrEdvG24h63HOxj+DLFK5babdXqfPibid3FhvHHmTAAAJ6QhAEA8ITh6OIExqzCr/wLv7IOlUNuz0NMPKGte4/UyYGN6Ns+tdOpY/Qz8XadeVjYI5HvR51f116hu/hVex/oQ5otc9oNOvAz+xxxL5m95rkbTNzk39+XpKsV1sZj3eHofzcYF7Ftr1kXmLj97XNNnOjh3aVvdDLxc11eiPp5i55pZ+LaWycX0zK+OBMGAMATkjAAAJ6QhAEA8IQ54eIEtk0Jv/w+eHn7nAdaOXXZ124UJI+UrCwTPzDSzgOHXxfw9XY7p6Snx3kNVRJKbbC/U952VAsT76przw9Szlkf1euN7Tg07JGMiG3nnvRMVK/Z7/deJp72SQenrvlj9o4/Jb+PV8W04Yyd+25UaPmKeibO3lryXedidXunT018aEbkGeh+y05wyvU+WWjiRM9bB3EmDACAJyRhAAA8YTg6HqpUlsGoyiG1Xl2nvP1Vu0l914zIO+48N+l4E7eRHxPTuQou96RDTZx1z1KnblzLESYOLgksbicmV/q+mxQKDjP/cUvTyA1/mGHCpuIuQ6qMn/q7unzilIu7aUN2v58S3R1j68d2SrBvzeDStMj9++25jk653h9ltywpiDNhAAA8IQkDAOAJSRgAAE+YEwbCLO/Xzin/dNCwItv9e30np9z+8bUmjuWuTJXB76far5wJLSc4da9sa2TizfnVTfzeqs5Ou3VfNZKiDO/3rFPuUc0uNOn288VOXd0+8wOlzcV3Gka+ds/bop+vL73U2vbajIXPNHPqZnd6Pqo+dXjzRhO3HuVnDjgcZ8IAAHhCEgYAwBOGo1EpBXfBEhFZ+0pDE7/T+eGw1lVMNHyTHaqe8NCxTqtai3+IXweTVO05dhe67I+uc+raD7ZDxPmbt5i4ivzutGscVt7r10vcIcrjqi6IuZ/wL3jHMhGRBvfZ4zmu6Ziw1kWfT36+y/2ctx1ldzMsy12xisOZMAAAnpCEAQDwhOFolFtrBh1l4io93U38H+3wpolDOrrfkvcvPc3EQ1q869QFd8IKDj+H++pCu+NTraL+eqQAAAP+SURBVNkMP5dU/ZGTA7Fbl8jhwfSX6+67EeJqw9VHmrje6OiuRJ7/vB2CbtZog1M3qukXJe7DjR9f4ZTb/Fb+drLjTBgAAE9IwgAAeEISBgDAE+aEUW5su/AIp/zT4Ccitk1XqSbO1blRvf5H7ew8cPD5Ba9h4y2h3U5dj0cGm/iA2e6ddOBXaoP9TdwwveilSyIiabsr4z2P4u+hGSc55b7HPB+hpUiHfrNN/NOB9vqO/hd95LS7vvYiE6erX0ycq8OvEoh8zhj8PGePvcHEbf5ePnbFKg5nwgAAeEISBgDAE4ajw6g0+0+SUWOPx55UPqt7ubc9KG4j9uDwcSybyAefH/4a/7emh1PX6NM/TFxedtlBgW1HtTDx2ZkfhtVyjhFvjUemO+XJ3eww8OEZ7rSQs6TousjLi4Kf3mg/18Gd60RERo23w+Qt//mzicM+5uUSf6UAAHhCEgYAwBOSMAAAnjAnHCalRVMT/3LUcxHbBZexNPyIf8ZYpdaz2wlefMgUjz2xHm/4jVP+anymiZ84pruJ89asLasuIQopYecUwc9o2nZm8+Mh7YtpTvm2IQNM/M0Dw+P6XivycpzyQ2t7mXj5lU2cuha/2aVIFWEeOIgzYQAAPCEJAwDgCeOo4TZuNuHBL95k4kOPm+s0W/FwGxNnvlv+7sxRUeR2sDdiv3f/CXF//VN+O8/Eayc1shXKbXfnpfauTBdmrXbqTqi23cRPZES+wxL8Cl/S8uKWg02c/vm08OaIg/qf2N2uuja52ambPmBYqV777GG3O+UDHwvuVje/VK9dnnAmDACAJyRhAAA8YTg6TP6GjSZuEdj8e0NYu2pSPq7krejSV9vh/2OmX+rUfdv1lYjPW52/y8S9XrI3WGg9coXTLmOVHVpukht5g//Xn+lq4jeqH+nUbT2koYmztibPMFiyGzXnaBM3lZkee5K88teuM3GTf69z6s74d7dSvfaBUjlulsKZMAAAnpCEAQDwhCQMAIAnzAnDq/yFS0xct49bd4ZEN6fUXOzcfV4x7Yrtxx9/RKyr/vty2y7G10dirOwZuS7zo8zIlUA5wZkwAACekIQBAPCE4WgAFVbKbrv12aMbDnLq6j4/Obw5UO5wJgwAgCckYQAAPCEJAwDgCXPCACqsVrf+YOJJUs1jT4DYcCYMAIAnJGEAADxRWmvffQAAoFLiTBgAAE9IwgAAeEISBgDAE5IwAACekIQBAPCEJAwAgCckYQAAPCEJAwDgCUkYAABPSMIAAHhCEgYAwBOSMAAAnpCEAQDwhCQMAIAnJGEAADwhCQMA4AlJGAAAT0jCAAB4QhIGAMATkjAAAJ6QhAEA8IQkDACAJyRhAAA8+X8tgscturQpyQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize=(8, 8))\n",
    "\n",
    "for idx in range(16):\n",
    "    plt.subplot(4, 4, idx + 1)\n",
    "    plt.axis('off')\n",
    "    plt.title('[{}]'.format(np.argmax(mnist.train.labels[idx])))\n",
    "    plt.imshow(mnist.train.images[idx].reshape((28, 28)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，定义用于训练的网络，首先定义网络的输入。\n",
    "\n",
    "这里我们直接使用上面的数据作为输入，所以定义两个placeholder分别用于图像和lable数据，另外，定义一个bool类型的变量用于标识当前网络是否正在训练。\n",
    "\n",
    "为了让网络更高效的运行，多个数据会被组织成一个batch送入网络，两个placeholder的第一个维度就是batchsize，因为我们这里还没有确定batchsize，所以第一个维度留空。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.706044Z",
     "start_time": "2018-06-01T06:32:51.698913Z"
    }
   },
   "outputs": [],
   "source": [
    "x = tf.placeholder(\"float\", [None, 784], name='x')\n",
    "y = tf.placeholder(\"float\", [None, 10], name='y')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因为我们输入的是图片展开后的一维向量，所以第一步就需要先把一维向量还原为二维的图片。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.719298Z",
     "start_time": "2018-06-01T06:32:51.707730Z"
    }
   },
   "outputs": [],
   "source": [
    "x_image = tf.reshape(x, [-1, 28, 28, 1])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:07:36.530623Z",
     "start_time": "2018-06-01T06:07:36.522665Z"
    }
   },
   "source": [
    "接下来，我们定义第一个卷积层，使用6个5X5的卷积核对输入数据进行卷积，\n",
    "padding方式选择valid，所以输出数据的宽高变为24x24,但是深度已经从原来的1变成了6。\n",
    "本层卷积的激活函数为relu。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.764292Z",
     "start_time": "2018-06-01T06:32:51.721295Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\chen\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\framework\\op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('conv1'):\n",
    "    C1 = tf.contrib.slim.conv2d(\n",
    "        x_image, 6, [5, 5], padding='VALID', activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来进行stride为2的最大池化，池化后，输出深度不变，但是长宽减半，所以输出变成了12x12,深度6."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.774112Z",
     "start_time": "2018-06-01T06:32:51.766784Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('pool1'):\n",
    "    S2 = tf.contrib.slim.max_pool2d(C1, [2, 2], stride=[2, 2], padding='VALID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:10:16.678485Z",
     "start_time": "2018-06-01T06:10:16.671472Z"
    }
   },
   "source": [
    "接下来，我们定义第二个卷积层，使用16个5X5的卷积核对输入数据进行卷积，\n",
    "padding方式还是选择valid，输出8x8,深度为16，本层卷积的激活函数为relu。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.805912Z",
     "start_time": "2018-06-01T06:32:51.776959Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('conv2'):\n",
    "    C3 = tf.contrib.slim.conv2d(\n",
    "        S2, 16, [5, 5], padding='VALID', activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再进行一次stride为2的最大池化，输出为4x4,深度16。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.814748Z",
     "start_time": "2018-06-01T06:32:51.807560Z"
    }
   },
   "outputs": [],
   "source": [
    "with tf.name_scope('pool2'):\n",
    "    S4 = tf.contrib.slim.max_pool2d(C3, [2, 2], stride=[2, 2], padding='VALID')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "池化后的数据是3维的，这里做一个拉平的操作，将3维数据展开到1维，然后送入两层全连接，全连接隐层中神经元个数分别为120，84。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.856740Z",
     "start_time": "2018-06-01T06:32:51.817287Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\chen\\Anaconda3\\lib\\site-packages\\tensorflow\\contrib\\layers\\python\\layers\\layers.py:1624: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use keras.layers.flatten instead.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('fc1'):\n",
    "    S4_flat = tf.contrib.slim.flatten(S4)\n",
    "    C5 = tf.contrib.slim.fully_connected(\n",
    "        S4_flat, 120, activation_fn=tf.nn.relu)\n",
    "\n",
    "with tf.name_scope('fc2'):\n",
    "    F6 = tf.contrib.slim.fully_connected(C5, 84, activation_fn=tf.nn.relu)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:17:39.880958Z",
     "start_time": "2018-06-01T06:17:39.869797Z"
    }
   },
   "source": [
    "###### 对特征添加一个0.6的dropout，以40%的概率丢弃特征中的某些数据，\n",
    "这样可以提高网络的推广能力，减少过拟合的可能性。\n",
    "\n",
    "需要注意的是，dropout仅在训练的时候使用，验证的时候，需要关闭dropout，\n",
    "所以验证时候的keep_prob是1.0。\n",
    "\n",
    "dropout的输出最终送入一个隐层为10的全连接层，这个全连接层即为最后的分类器。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:51.913419Z",
     "start_time": "2018-06-01T06:32:51.860766Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-15-413a88f1eb86>:3: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.\n"
     ]
    }
   ],
   "source": [
    "with tf.name_scope('dropout'):\n",
    "    keep_prob = tf.placeholder(name='keep_prob', dtype=tf.float32)\n",
    "    F6_drop = tf.nn.dropout(F6, keep_prob)\n",
    "\n",
    "with tf.name_scope('fc3'):\n",
    "    logits = tf.contrib.slim.fully_connected(F6_drop, 10, activation_fn=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来定义loss和用于优化网络的优化器。loss计算使用了sparse_softmax_cross_entropy_with_logits,\n",
    "这样做的好处是labels可以不用手动做one_hot省了一些麻烦。这里使用了sgd优化器，学习率为0.3。\n",
    "\n",
    ">试试看，增大减小学习率，换个优化器再进行训练会发生什么。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:52.084738Z",
     "start_time": "2018-06-01T06:32:51.915376Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From <ipython-input-16-3c77ec77605a>:2: softmax_cross_entropy_with_logits (from tensorflow.python.ops.nn_ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "\n",
      "Future major versions of TensorFlow will allow gradients to flow\n",
      "into the labels input on backprop by default.\n",
      "\n",
      "See `tf.nn.softmax_cross_entropy_with_logits_v2`.\n",
      "\n",
      "Conv/weights:0\n",
      "INFO:tensorflow:Summary name Conv/weights:0 is illegal; using Conv/weights_0 instead.\n",
      "Conv/biases:0\n",
      "INFO:tensorflow:Summary name Conv/biases:0 is illegal; using Conv/biases_0 instead.\n",
      "Conv_1/weights:0\n",
      "INFO:tensorflow:Summary name Conv_1/weights:0 is illegal; using Conv_1/weights_0 instead.\n",
      "Conv_1/biases:0\n",
      "INFO:tensorflow:Summary name Conv_1/biases:0 is illegal; using Conv_1/biases_0 instead.\n",
      "fully_connected/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected/weights:0 is illegal; using fully_connected/weights_0 instead.\n",
      "fully_connected/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected/biases:0 is illegal; using fully_connected/biases_0 instead.\n",
      "fully_connected_1/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected_1/weights:0 is illegal; using fully_connected_1/weights_0 instead.\n",
      "fully_connected_1/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected_1/biases:0 is illegal; using fully_connected_1/biases_0 instead.\n",
      "fully_connected_2/weights:0\n",
      "INFO:tensorflow:Summary name fully_connected_2/weights:0 is illegal; using fully_connected_2/weights_0 instead.\n",
      "fully_connected_2/biases:0\n",
      "INFO:tensorflow:Summary name fully_connected_2/biases:0 is illegal; using fully_connected_2/biases_0 instead.\n"
     ]
    }
   ],
   "source": [
    "cross_entropy_loss = tf.reduce_mean(\n",
    "    tf.nn.softmax_cross_entropy_with_logits(logits=logits, labels=y))\n",
    "\n",
    "l2_loss = tf.add_n([\n",
    "    tf.nn.l2_loss(w)\n",
    "    for w in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES)\n",
    "])\n",
    "\n",
    "for w in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES):\n",
    "    print(w.name)\n",
    "    tf.summary.histogram(w.name, w)\n",
    "    \n",
    "total_loss = cross_entropy_loss + 7e-5 * l2_loss\n",
    "tf.summary.scalar('cross_entropy_loss', cross_entropy_loss)\n",
    "tf.summary.scalar('l2_loss', l2_loss)\n",
    "tf.summary.scalar('total_loss', total_loss)\n",
    "\n",
    "optimizer = tf.train.GradientDescentOptimizer(\n",
    "    learning_rate=0.3).minimize(total_loss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:25:56.449132Z",
     "start_time": "2018-06-01T06:25:56.438340Z"
    }
   },
   "source": [
    "需要注意的是，上面的网络，最后输出的是未经softmax的原始logits，而不是概率分布，\n",
    "要想看到概率分布，还需要做一下softmax。\n",
    "\n",
    "将输出的结果与正确结果进行对比，即可得到我们的网络输出结果的准确率。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:39:50.010829Z",
     "start_time": "2018-06-01T06:39:49.997501Z"
    }
   },
   "outputs": [],
   "source": [
    "pred = tf.nn.softmax(logits)\n",
    "correct_pred = tf.equal(tf.argmax(y, 1), tf.argmax(logits, 1))\n",
    "accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "saver用于保存或恢复训练的模型。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:32:52.127795Z",
     "start_time": "2018-06-01T06:32:52.103115Z"
    }
   },
   "outputs": [],
   "source": [
    "batch_size = 100\n",
    "trainig_step = 1100\n",
    "\n",
    "saver = tf.train.Saver()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上定义的所有操作，均为计算图，也就是仅仅是定义了网络的结构，实际需要运行的话，还需要创建一个session，并将数据填入网络中。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:35:22.270272Z",
     "start_time": "2018-06-01T06:33:18.829198Z"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "after 100 training steps, the loss is 0.240914, the validation accuracy is 0.898\n",
      "after 200 training steps, the loss is 0.159011, the validation accuracy is 0.9468\n",
      "after 300 training steps, the loss is 0.185066, the validation accuracy is 0.966\n",
      "after 400 training steps, the loss is 0.0639687, the validation accuracy is 0.9768\n",
      "after 500 training steps, the loss is 0.0627127, the validation accuracy is 0.9738\n",
      "after 600 training steps, the loss is 0.0751049, the validation accuracy is 0.9776\n",
      "WARNING:tensorflow:From C:\\Users\\chen\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\training\\saver.py:966: remove_checkpoint (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to delete files with this prefix.\n",
      "after 700 training steps, the loss is 0.10243, the validation accuracy is 0.974\n",
      "after 800 training steps, the loss is 0.0946659, the validation accuracy is 0.978\n",
      "after 900 training steps, the loss is 0.0795162, the validation accuracy is 0.9814\n",
      "after 1000 training steps, the loss is 0.166075, the validation accuracy is 0.9754\n",
      "the training is finish!\n",
      "the test accuarcy is: 0.985\n"
     ]
    }
   ],
   "source": [
    "merged = tf.summary.merge_all()\n",
    "with tf.Session() as sess:\n",
    "\n",
    "    writer = tf.summary.FileWriter(\"logs/\", sess.graph)\n",
    "\n",
    "    sess.run(tf.global_variables_initializer())\n",
    "\n",
    "    #定义验证集与测试集\n",
    "    validate_data = {\n",
    "        x: mnist.validation.images,\n",
    "        y: mnist.validation.labels,\n",
    "        keep_prob: 1.0\n",
    "    }\n",
    "    test_data = {x: mnist.test.images, y: mnist.test.labels, keep_prob: 1.0}\n",
    "\n",
    "    for i in range(trainig_step):\n",
    "        xs, ys = mnist.train.next_batch(batch_size)\n",
    "        _, loss, rs = sess.run(\n",
    "            [optimizer, cross_entropy_loss, merged],\n",
    "            feed_dict={\n",
    "                x: xs,\n",
    "                y: ys,\n",
    "                keep_prob: 0.6\n",
    "            })\n",
    "        writer.add_summary(rs, i)\n",
    "\n",
    "        #每100次训练打印一次损失值与验证准确率\n",
    "        if i > 0 and i % 100 == 0:\n",
    "            validate_accuracy = sess.run(accuracy, feed_dict=validate_data)\n",
    "            print(\n",
    "                \"after %d training steps, the loss is %g, the validation accuracy is %g\"\n",
    "                % (i, loss, validate_accuracy))\n",
    "            saver.save(sess, './model.ckpt', global_step=i)\n",
    "\n",
    "    print(\"the training is finish!\")\n",
    "    #最终的测试准确率\n",
    "    acc = sess.run(accuracy, feed_dict=test_data)\n",
    "    print(\"the test accuarcy is:\", acc)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2018-06-01T06:58:16.766415Z",
     "start_time": "2018-06-01T06:58:15.918700Z"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From C:\\Users\\chen\\Anaconda3\\lib\\site-packages\\tensorflow\\python\\training\\saver.py:1266: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Use standard file APIs to check for files with this prefix.\n",
      "INFO:tensorflow:Restoring parameters from ./model.ckpt-1000\n",
      "1.0\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAeEAAAHiCAYAAADf3nSgAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvIxREBQAAIABJREFUeJzs3XeYFMXWBvD3bGAJS855yVFRwYgKqIheRTEHBFHR6zXrVa8BL5gw3M+cE2IWBXPAgKIgAgICSlRJgmTJYYHd8/1Rvd1dw/QyO2F7w/t7nn329FRNdffUdNd0VQdRVRAREVHxSwt7AYiIiMorNsJEREQhYSNMREQUEjbCREREIWEjTEREFBI2wkRERCFhI0xERBSSUtcIi8gwEdktIltFpEqM7/lDRHaJyOuF5FER2SYi9yZvaYufiHwjIjtFZGLYyxIr1mnhSludsj4LV9rqM5KIjHTqakmM+ds634U8ERkckKeniOQ7+U5I6gInmYhc4iynikjrRMsLpRF2VsD/lyciTxShiFGqmq2q25zyPo8ob5eI/FKQWVVbARgeQ7ldVPV233I+LyILnC/HoCjrcb2IrBKRTSIyQkSyfGk5IvKtiGwXkfkiclzQTEUky3n/Zqe8G3xpTUVksoj8LSIPRbxvrIh087+mqscAuDyGdU0aZ/lfEpGlIrJFRH4WkROLWExknd4kIr865S0WkZv8mVmnqSUiV4nINBHJFZGRcRQRWZ+9nM9uU7SdN+uzeIlILRF5X8yPmqUicn4Ri3hQVXN85QV+Pqq6UFWzAUzYR5l/Od+ZsU6ZIiK3i8gyp9y3RaSab56NReRD53NfLiKFfqYicr6zrttE5AMRqeVLe1RENojIjyLS2Pd6fxF5zF+Oqr7krE9ShNIIOx90trMi9QHsAPBuAuWdGFHmpETK85kF4AoAMyITRKQPgFsAHAsgB0BLAHf6srwF4GcAtQHcDmC0iNQNmM8wAG0ANAfQC8DN4v0avBXAKwBaAOhXsEGLyDkAFqnqtPhXL2kyAPwJoAeA6gDuAPCOiOQkUKYAGAigJoATAFwlIucmtpgAWKex+gvAPQBGJKm8bU5ZN+0rYxGxPuPzFIBdMPvf/gCeEZFOCZQ3DMGfT7wGAhgAoDuARgAqAfAfrL0OYDHMOpwEYLiI9IpWkLNuzznl1QewHcDTTtohALoCaABgIkx9QkSqA7gRwH8TXI/CqWqofwAuBLAIgMSYfxiA1wtJzwGQB6BFEd+nAFoHpE0EMCjitTcBDPdNHwtglRO3BZALoKovfQKAywPKXwHgeN/03QDeduLPAbRz4rcBnA2gGszOo0ZAeYMATAy5XmcDOCMZderkeRzAE6zTYq/HewCMLOJ7AusFwHEAlsTzPWB9Jq1Oq8A0wG19r70G4P4Y3z8SwD2xfj6+18YDGBxQZk8AyyNeGw3gJt/0EQB2AqgMINv5PtT1pT8P4LWA8ocDeNM33cr5DKoCOAfAfc7rJwD4zImfBHB+PN/HovyVhDHhCwG8qs5aAYCIbBSRI+MsbyCACaq6OClLF6wTzK/wArMA1BeR2k7aIlXdEpG+1y9NEakJ8ysvsqyCvL8C6C0iNQB0AzAX5gv+qKpuTNK6JJWI1IfZyc3xvRZ3nYqIADjKX16KsE5jlOA2WlxYn9G1BZCnqgt9r7nrIyLNnPptFkthMXw+8RLnzz+dBXPELb7X/OmdA8qyvguq+gecHyIw+5WjRKQSzA+1OU5vRjtVfTPBddinUBthp5J7wHTluFS1hqrGe9LCQJhfaqmWDWCTb7ogrholrSC9akA5/vdH5r0PpgH6DqYLKRPA/gA+FpE3ReR7Ebkq3pVINhHJBPAGgFdUdX7B6wnW6TCY7+rLiS9hoVinMUqwPosL6zO6QtddVZc59busCOUVlLFXeQn4HMBgZ+y+OoD/OK9Xdn48/QDgDhGpKCIHATgD5ig5aBmjrrOq/gpgDIDJAJoBeADAYwCuEZFrnPp7w/mRlXRhHwkPhOmSScpRq/PLvAFMN0Zh+fwncvWPc3ZbYbqcChTEW6KkFaRvwd62Rrzfyquqf6vqOaraBeaL8QSAq2HGun6F6d67XEQ6xrkeSSMiaTDdWrsAJGWn4+y8BgI4SVVzC8nHOi1DWJ8pVZR1j7W8gjJiKk/sE2mDjrhHwIzbj4c5Wv3WeX25878/zDj8nwCegfnxvxzRFbrOqvqIqnZR1XNguqcnwLSPl8EcHc+Dqc+kKwmN8Cv7zBW7CwG8p6pbC8uk9olcb8Q5rzkAuvimuwBYrarrnbSWIlI1In2v7lRV3QBgZZSyonW9XgZgsvPLbT8A01R1F4BfENwNUyycLuOXYE56OENVdyehzIvhnFijqkEbFwDWaVnD+kyphQAyRKSN77Wg9dmnIn4+Be/J9v1FPeJW1XxVHaqqOaraxClvhfMHVV2qqieral1VPRTmBLupAbO0vgsi0hKma9vfJV8wlPZPAHfB1NdsZ1/2E0zvRtKF1giLyBEAGiM5ZzHD6c8/C0nsihaRCiJSEWasIdPp9ij4zF4FcImIdHTGRIYUzNsZa5kJYKjzntNgKnBMwKxeBTBERGqKSHsAl0auh4jUA3AlTNcsYM4K7CUi2TDjUIuSsMqJeAZABwB9VXVHooU5Rz/DAfRW1aStG+s0NiKS4XxO6QDSnXXOSKC8NKe8TDMpFUWkQhKWk/VZRGouG3sPwF0iUkVEugM4FaYXK177/HyKSsxlVK3E6AjgYQB3qWq+k95BRKo634ELABzv5InmDQB9ReQoMdeu3wVzwBZ5tP4wgKGquh2m/g526q8nUlV/iZ7ZFe8fzOniQWeybQVwVEDaMEQ5gxLAeQCWIuAs66D3+dL3OtMNphtEI/56+tJvALAawGaY8cosX1qO8/4dABYAOM6X1h/AHN90FkzXy2anvBuiLN+rAM7yTTcFMAXABgAPReQdhGI88xLmsgSFOXNxq++vf7x1CrMB7I4o71nWabHV6bAon9OwBOqzZ5TyxrM+i6c+oyxrLQAfwFw6tgy+s4BhxkW3AmgW8N6R2Pvs6Fg+n/Eo2tnRbZ162Q6zb78hIv06AGuddZgIoFtEuvUdBXC+s67bAHwIoFZE/l4APo147VGn/iYDaLKv72NcdRHWlyCBL88Q50PcCKBKjO9Z4FTIiELy7IQZqL877HVM8PP5CmacY1zYy8I6LZ91yvosW/UZZflfcOrqjxjzt3G+C9sRcRmZL8/RMD+GNgLoE/Y67mN9LnKWcyeAlomWJ06hREREVMzCPjGLiIio3GIjTEREFBI2wkRERCGJ+5KDePROO4sD0CH5Kv9d2XeuomF9hicV9QmwTsPEbbRsibU+eSRMREQUEjbCREREIWEjTEREFBI2wkRERCFhI0xERBQSNsJEREQhYSNMREQUEjbCREREISnWm3UQpcKSew5347yK9r0J6nZa68Y/dgl6VCzQ6puL3Ljq1EpWWv3HJyW6iEREUfFImIiIKCRshImIiELCRpiIiCgkHBOmUmnDp23c+NcDnozpPbsLuZX9/F4vuvEb3Rpaae981cON8+b9FuMSUkkiXTtZ059+9Job7/fsVW7c9G6O/xe39BrV3XjBky3d2L9NAsCQNV3d+Jf+ba20vLkLU7R0qccjYSIiopCwESYiIgoJu6OpVPB3PwPADwe8HdP7nt3odW89/GNvN85pvtbK92XH99y4f9WVVtq9g+q4ccv/sDu6NFpzcDVreg/y3LjyX3zkbpjyWzRx4196PufGkcNH99Sb7sZdTjvCSmvK7mgiIiIqKjbCREREIWF3NJVYe471zob8pstTEamZbvToBu9MyW/P6WZn+2uNG7bdMM2N0ypWtLINn7KfG99W5xd7OWruiXmZqWTasH+eNb18T64b137px+JenHIto2kTa7rF87+HtCQlA4+EiYiIQsJGmIiIKCRshImIiEJSqseE1196uDXdbIA3tjB/TX0rbVeuN4bY+C0vrrx8q5Uvf+bcZC4iJWBr4wpunBbxe9E/Djz+FG88N2/RgpjK/v3OA63pN2s95JvKstKajOVv1dJIux/gxhNOfthK6/H91W7cGj8X2zKVV8v+611S1PUEex/7YMMJRS4v+wj7EsM/7/DKrzPbO4ej0odTi1x2cePehYiIKCRshImIiEJSqrujb77pTWv6jCobvIlWhbyxpxcu2bPdSnpsba/EFyxGU9c0d+MqD1W30jLGTY/MXu7UeNW7dOTMaRdYabJhsxvvWbmkyGUP/sfX1nR2WlZATiqt/u5YyY0bple20hqPzozMTik0+59PuPFuzSskZ2zGd3nDfqGLF76/zXsAy4gt/axsGd+UvP0qj4SJiIhCwkaYiIgoJGyEiYiIQlKqx4Qfv+1ca/q/+3u/KWrOsx/BsaGDuHGF/Te68YOd37PyPdJwiht/uj3bjU+qbF/KVJgdusuNp+RWceOeFXfbGX3zan3OP62ktuNinl25kIyHdi+517uk7ZIa/xeR6t3G8t8rD7NSqn49z1uOhJeCisuxV3jnFHywrYaVlj3eu5SNdZoameO9sdlMSU+4vJ935bvxkt11rbTTqvztxmdne7eqPfu15618JzfuipKGR8JEREQhYSNMREQUklLdHV1l9JSI6eC81QJef6JBT2v6nu453nu+8+7A9WDP1jEvV8YOr9ukymzvAfG1vx9j5duvgu/OXUt4yUQqbBzgdUH/MNDrgq6eZj9F6cdcr7ts5j323bQqbS75d90hIL1TO2t6eL233PilzfaTe/I2biqWZSpPdvQ7xJq+qOG7buy/LCnWS5Q6j7vcmq47zruMMGuTXcatPb3jyV/OejywzOW3enfWanLfpJiWI9V4JExERBQSNsJEREQhKdXd0cmwZ9Vqa7rKGG/a3+FRZfT6uMpfPdjrDu1Uwf64/+9vr/ss5+VF9nLFNTeKtO4g7yz5yC5ovwvHD3bjth+w+7k0WtG7dmDa9C3NI17ZkdqFKSf8QwD3PGyfidytwi5/zsAy/He4GvLtGW7c4eb5Vr68zZsRpN1v3gNdpp7ibeeHZO208n3+rwfd+PiKN1tpOcO9u2lpbm7gvJKNR8JEREQhYSNMREQUEjbCREREISn3Y8KpkNG8qRs/eduTbhx515h3HzvOjWuv/BGUuF1f2WN/P7Z/yDfljRV1+fFCK1+Hf//hxryDUum0uePuwLSZTx5gTdcAt7dkyPed52KPAQe7eOkJ1vSWc7ynXbVd7p2PUZTt0H9HvStGepc2Tfvno1a+hunevGZcYqed8Z63T9BZ81BceCRMREQUEjbCREREIWF3dArMv76xGx+c5T04Ys4u+7KIWnO3F9sylWUZLXPc+O7W71ppNX2XJU33XXXQ/G67sytvw4aULBulVu6JB7vxh8c/YaXdtc67WX+tMbOttHxQcbptdTc33jzYvpQsb/lvSZ1Xzph1bnxHP/thLPc3+Cmp80oGHgkTERGFhI0wERFRSNgdnQS5Jx1sTc848xHflHfT8X9de62Vr9Ik3pkpGVq9s8KND6wQ/LvyPN8N4dvOKnndUlR0y4/xdmH7V7DviHbhkv3cuN42++5LlHyFPTN49kH+57snt/t5L+INAWak2QMPhS3jX3d6cYN+SV+qQDwSJiIiCgkbYSIiopCwESYiIgoJx4STYNmJ9m+ZbPHGgc9b3NuNK4+dZeVTULw2XOg9nerO+v67YmVZ+S5c4t2VrMPNv7sx74pVNtTtvMaN89Qe/8v4sGZxL065s+Bfld14t5aMrWrJ6d4lUKPr2ufd7NZ0X2wvb6OhXlycl7DxSJiIiCgkbISJiIhCwu7oOKVVrerGA46aaKVtzvceJL1meEs3zsrlZTHxymjcyJo+6popbpydlhWZ3fXj3NZu3HYDP/+yIKOF95CO/2vn3SHthU1NrXy1RvAhDak25KiPQ5lvRtMm1vSWrt7+4dmLno6pjKm59iVtsmtP4gsWBx4JExERhYSNMBERUUjYCBMREYWEY8Jx+m1YJzf+pI49BnHqb2e4cdZnHIdMhnm32eN9HzSIPhbV65ezrGlellT2/PZPb/zvMN/pAJfO6GXla4pfi2uRqJjNvbOBNT3n+Cdjet+YrXXc+Jkb7X1FxXnh3EaYR8JEREQhYSNMREQUEnZHx2jTBfbDoWef87gb/7Fnt5W29QHv9PksrEztgpUT0095JOKV6JclVb/CvtfNng0bUrREFJb8pjujvr5jY8Wor1PZkDm+oRvf13BMXGWMXHGEG1f8uGQ8xY5HwkRERCFhI0xERBQSdkcXwn+XpuvuGGWlZYn30Z07a4CVVvdznhEdlt31q1vTmbsaF7mMvLXrrGnNzXVjyfK6wdPr1kGQvLo1rOnf/l0hpnlrnvdA8vZX/26l5W3eHFMZZd3Th74e9fXGnwc/sJ1SI1284Z9MCf78N59/WGDanXe95Ma9KkUfaogsf++HRcRW93rMipjyFSceCRMREYWEjTAREVFI2AgTERGFhGPCESTD+0i6fLLcjc/KXm/le2NLPTeuf4f9W6Y4HwhNtk9Hj0i4jCN+Ps+aXre6mhvXrLvFjad0fTPheRWm45CrrOmWN5fPpwLt7HuINX1kRf+lJdyFhen+UWe68dmXPBqY7/v/PeXGe4/nwpcW23wLK8Ov87jLrek2mBHbDIoRj4SJiIhCwkaYiIgoJOzLidSlnRveXe+1wGxPDfdu/l1jVvnsJixOp87tb02P6zw6ZfOadOBbcb1vu+5y490aPCjxj9mD3HjTzODLnBpPDOch4yXNslPsPkr/5YF3rdvPjbM/nG7li7FnkxLQcpR3Od/UC+w7lh2SFXy5UaKm5trzen5VDzfecIX3cIf2iyMu80vZEsWPR8JEREQhYSNMREQUEjbCREREISn3Y8LpHdta05e9/WHUfB1HXGlN57w2OWXLRHur1GexNd1puHf5jsb4La7a/m83LsrlRZ0mXOTNa1mVwHwtR2/1Jqb+EpivJn6LGpMnvZp3Wdh/un8WmO/Nz49245Z7eG5Gccubu9CN/3vDYCvtz77eeRELT3wuqfO9YoR96VHTeyf5pkrXk9N4JExERBQSNsJEREQhKffd0fOvqGlN960c/Uk1Tcbvsl9QXgARpha3Jdb1eDK6xj4vzE5oXlR0+b4nV83d3shKO25FNzduM3yOG5fEy0/Kk0ofTrWm2/pG9o4+zxvOyxy02so3tpP3hLrjfz3XjfNH1rPyqfeAMeTMXGullea655EwERFRSNgIExERhaRcdkf7bwg/ru9DEamVi3dhiGgv6uuOXtDNTquApW5cmrshy5Nqb/muJom4Id1p8PbHVbDIl7IIQcpSvfNImIiIKCRshImIiELCRpiIiCgk5XJM+K/u6W7cLCN4DPiNLd4p8pmb7UuUeIESERElikfCREREIWEjTEREFJJy2R1dmPvWd3TjH/vkuLGuDL4hPxERUTx4JExERBQSNsJEREQhYSNMREQUknI5JtzyFu8JPP+45aBCcq5K/cIQEVG5xSNhIiKikLARJiIiCokoH05PREQUCh4JExERhYSNMBERUUjYCBMREYWEjTAREVFISn0jLCIjRWSXiCyJMX9bEdkqInkiMjggT08RyXfynZDUBU6yWNanpBORYSKy21mPKjG+5w+n3l8vJI+KyDYRuTd5S1v8ROQbEdkpIhPDXpZ4cBst/duoX3nfXpNdnyWqERaRNs7OJrCiAjyoqjlRyqslImv9Oy9VXaiq2QAm7KPMv1Q1W1XHOmWJiNwuIstEZLOIvC0i1XzzaiwiH4rI3yKyXEQuL6xwEblaRBY7ZU0TkSN9aeeLyEonvafv9VYiMklE3AciF2F9UkpErnLWI1dERsZRxCjn897mlNdLRL4VkU3Rdt6q2grA8BjK7aKqt/uW83kRWeDswAdFWY/rRWSVM98RIpLlS8txlmm7iMwXkeOCZioiWc77Nzvl3eBLayoik53vykMR7xsrIt0i1vUYAIV+n4pLKd9Ga4nIKBFZ5/y94U+PsmyDReR3Z4c7VkQa+dJK3TbqJyIdxPy42+Ss42lFLCJye60hIq+IyBrnb5g/cwLba18R+dWpg0ki0tGX9qzzesFfrohsKWSdCxr5gvwv+tJCq88S1QgDeArAT0ks7wEA85JU1kAAAwB0B9AIQCUAT/jSXwewGEB9ACcBGC4ivaIVJCKHArgfwJkAqgN4CcD7IpIuIhlO2kEArgbwpO+tjwO4QVXzkrROyfQXgHsAjEhSeducsm5KUnkFZgG4AsCMyAQR6QPgFgDHAsgB0BLAnb4sbwH4GUBtALcDGC0idQPmMwxAGwDNAfQCcLN4R2y3AngFQAsA/QoaXRE5B8AiVZ0W/+qlXGneRu8BUBOmXlvBbKvDohUkIj1gGo1TAdSC2bbfctJK6zYKwF3+DwF8ArNulwF4XUTaJlDsIwAqw2w3hwAYICIXJbicbQC8AfMDtAaAjwF85Cw/VPVy54dAttMovgXg3X0U28X3nsHOfEKtzxLTCIvIuQA2AhiXpPIOB9AZwMvJKA9AXwAvqeqfqroVZudxjohUFpFsAD0B3Kuqu1V1FoDRAC4OKCsHwBxVna7mQu1XAdQBUA9mB79CVVcC+BpmhwEROdN5fXKS1iepVPU9Vf0AwPoklTdVVV8DsCgZ5fnKfUpVxwHYGSX5Qpg6nqOqGwDcDWAQYLqgYDbSoaq6Q1XHAPgFwBkBsxoI4G5V3aCq8wC8UFAWTOP7japugmnQWjpHZLcAuC0Jq5kSpXkbddJbAPhAVTc7n/37ADoVUta7zndhF8x34WgRaYVSuo36tIf5kfKIquap6jcAfoD5AROvvjC9HdtVdQnMgUXQ/i9WfQBMUNWJqroHpj4bA+gRmVFMt/gZMD9uiyrU+iwRjbCzA7oLwL+jpDUTkY0i0qwI5aXD/GK/CkCy7kYizp9/OgvmaEd8r/nTOweU9TmAdBE51FnWiwHMhLlZ9VoAtUWkCYDeAOY4jfwQmCOoUsmpwyP3nTNUnWCOlAvMAlBfRGo7aYtUdUtE+l47cRGpCbOTiyyrIO+vAHqLSA0A3QDMhdnJP6qqG5O0LklVBrZROPM7WURqOnV0Bsy2GGtZgNmmS/s2KgGvufurOLfXWPd/RSkvssygcs+AqZfv91Hm92KGh94TkRzntVDrs0Q0wjA7oJdU9c/IBFVdpqo1VHVZEcq7BsAUVZ2etCU0G+tgMeOC1QH8x3m9srNj/gHAHSJSUUQOgvlSVA4oawuAMQAmAsgFMBTAZWrkA/gXzJH0jQAuhdn5PQFgPzFjkl+ISKJf8GLl1GFJP7EoG8Am33RBXDVKWkF61YBy/O+PzHsfgKMAfAfTMGQC2B/AxyLypoh8LyJXxbsSKVKqt1Hn/wwAFWB6a9YDyAPwdEBZnwE4W0T2F5FKAP4L82OhchnYRucDWAPgJhHJFJHjYY4u3f1VHNvrWAC3iEhVEWkNc2ARtP+L1VcAeog5Ca8CTC9RhYByLwTwqhZ+C8geML2Q7WGGzz4RkYyw6zP0RlhEDgBwHMyYQjLKawSzgd++r7y+9/gH94N+zY+AGXMYD2AOgG+d15c7//vDdHf9CeAZmLGM5YhuMMyXtBPMl+oCmC9EIwBQ1XGqepiq9gCQD3O0NBLAazBdmncDeHHvYssvEfncV4f94yxmKwD/iToF8ZYoaQXp0U4E2Rrxfiuvqv6tqueoahcAj8Fs7FfDdEf/CrM9XO4/CSVMZWgbfRfAQpgfQ9UA/AFzLsdenCGLoTA/lpcCWAJTf8sL0kvrNqqquwH0gzl3ZRVM78Y7CN5fxeIaADsA/AYz3vxWYeXFsr2q6nyYxvVJACthhuzmRpYrIk1hGthXC1tAVf1eVXc5vU3XwuyvOzhpodVnSXiUYU+YXyfLRAQwRxHpItJRVQt7zmCQQwA0BDDXKa8SgEoisgpA42gD7M6gvktEWkbJkw+zUQ518hwPYIXzB1VdCuBkXxlvApgasIxdAHysqgud6bEishLAETC/xgrKEJgv4DUwX8B0VV3qrMv+hX8M5YuqnpiEYubA1M07znQXAKtVdb2IzIEZu63q65LuAuDNKMuywanPLjC/5gvyzokyz8sATFbVX0VkP5hxul0i8gtMt9vcJKxXonqiDGyjMHVwhe+M3mdheqOiUtWnYHoqCs4JGALzI8m/HKVyG1XV2fCNrYrIJMQ3nlpQ3t8wByIF5Q1H8P4v5u1VVUfD2Sc6wzcXY+8TAwcCmKSqRT1/RBHRNR9GfYZ+JAzgeZgzFQ9w/p4F8CnMoHw8PofZYRSU91+YM1oPSOQMNzGXN7QSoyOAhwHc5Wz4Baf8VxWRCiJyAYDjnTzR/ATgJBFp6ZTXG0BbRGzgMEfMP6vqTJjus0rOvHshyScsJUpEMkSkIoB0mB10RXHOYoyzvDSnvEwzKRWdLqlEl7OCU64AyHTKLdgOXgVwiYh0dMYMh8D8Gobzg2kmgKHOe06D2SjHBMzqVQBDxIw/tofp4hoZsSz1AFwJ7wzdxQB6OeNR3VBy6rhMbKMw291gEankdDFfBnvc3l9WRRHp7JTVDOYzeMw5Yc+v1GyjfmK62SuKObH0RpgfRSMTKK+ViNQWc4XHiTCf7T1JWM6uTpl1ATwHc/AyPyLbQOxj2UWkk4gc4JSVDeAhmB9nkWfmF3t9ht4IO2fTrSr4g+nK26mqawH3pI/CuqAiy8uNKG8TgN1OnIg6MONE22B2IiNU9Xlfeh+YStoAc0r9CQXr4KzHVhE5ypl8FcDbMN1mm2FOg/+n/8slInVgukzucNZrD8xJLN/A7ASvTnB9km0ITHfULTDd6zuc1wDstf6xONop4zMAzZz4yyQs55dOWUfA7Fh3OPOCmutNH4Tpxlzq/A31vfdcmMZxA5xLzHzf0/7O0XKBoTDdnUthxn7/55Tv938wjURB9/V9AI6BGdL4SEvIpUplaBu9GKbxXw6zA24J74x1iMgcX9doRZhejq0wR3Q/wtkWfflL2zbqNwCmi3cNzCV5vVU1tyAxju21K8zVAltgvsf9VTVaz09RPQZzRv4C5/+l/kQxZ9g3QZRLk8R0eRdcbVAfwCiY/e0imO9vpVvcAAAgAElEQVTByU7XfEH+cOpTVUv1H8ylH1sB/BFj/jYwlbkdwKCAPAUNwEYAfcJex0TXp6T/wTTW25z1qBLjexY49T6ikDw7YXbwd4e9jgl+Pl/B7NzGhb0scS4/t9FSvo1GrE+53l6TXZ98njAREVFIQu+OJiIiKq/YCBMREYWkWC9R6p12Fvu+Q/JV/rvR7pKTENZneFJRnwDrNEzcRsuWWOuTR8JEREQhYSNMREQUEjbCREREIWEjTEREFBI2wkRERCFhI0xERBQSNsJEREQhYSNMREQUEjbCREREIWEjTEREFBI2wkRERCFhI0xERBQSNsJEREQhYSNMREQUEjbCREREIWEjTEREFJKMsBcgDHm9DnLjq55/x0p7pk3rlM13yzmHWdM1Zq7zlmnB7ymbLxXNxoGHW9NT7n/GjTs+dYUbN3tgqpVP9+xJ7YKVcRnNm7pxvVEb3fi76R2tfO2f9tLy5ixI/YI50uvWtabXn+jtK2qOmuHGmptbbMtEpR+PhImIiELCRpiIiCgk5bI7emmfLDeulb612Oa76qRd1vTuAd5voFonF9tiUBQZjRu58d3/fTEw39wrn3bjEx8/ykrTLVuSv2BlWEaD+tb0XePHuHG7zHw3PmZ9Aytf3pzfUrtgPv4u6P4TZ1hph1V8342v/OWfXsLPc1K+XKVZep3a1vSCR5q5cc82Xt2u6LHbyldWu/l5JExERBQSNsJEREQhYSNMREQUknIzJiyZFdz4mGNmhrIMVX+uaE2ffcl3bvxtjSZWWt7GTcWyTGSs6dPcjY+vvDsw30HTznHjulsXpnSZyqKMJo3duPqo7Vba/hXS3bjd15e7cZsL7bHY4jTvnhw3Pjt7rJV20KM3u3GjnycV1yKVSmuuOsKNh177qpV2UuUvo76nX52+1vSeFX8lf8FKAB4JExERhYSNMBERUUjKTXf0ltO8u2Q93vgJN+7wwVVWvjaYkrJlyK2p1vQ1Nee78fiqHezM7I5OqbTKla3pPtdMjOl9WW/X9CZUgzNSVBu6e3fF+iDnqcB8HYascePivA+ZHt7Fmv795OfcuMcvZ1lpTUd4229eaherVEpv28qNX/z3o258QAW72clHdCufqWpNN/ynd6nanpWrEl/AEoJHwkRERCFhI0xERBQSNsJEREQhKbNjwtr9AGv6qQcec+PXN3uXo7QfYl9mksqxncOP/zWFpVNR5B5hj8HfU++lwLzb873bjVZ7c3LKlqks8j8ZCQDWnrozMG+3/7vajRv8WXyX/PjHgYe88Upgvq2f2rfPrLJ+UcqWqSyYd4t3/oT/8rNYTen6pjW98EdvOzz9tRustJb3/uzG+TuDv2MlEY+EiYiIQsJGmIiIKCRltjt6w6323XiaZHgXOtxw9UlunLlhekqXI6Oh14X1cjP7jju7lb+BwrL49Ni7x878rZ9vqmzetSdV/nws25r+7ZCRbjxkjT1k1Phl7+lDxXnJz4qeVdy4e5Z9wUznSRe6cbMneFeswqR3bGtNf33so76pSm70wHp7KGjaRu8pSqNa2ftIv7a+ux6+0P8ZK+2BEae6cf7ipTEtb0nBVoCIiCgkbISJiIhCUqa6o9dfergbv7vf/6y0Vzft78aZX6e2C9pv7l3e2aG71e5ku3DJcW6ct2ZtsS0TAScdPCswbVP+Dmt69zDv4fNp7I4uElWxpv3bwJT1OVZa+o41SJW0qvbdlxbc29GNPzjlYTfOR6aVr9lZv6RsmcqadYfUtqZzMry70l3259FuvPywrVa+tCre0GHXy70z5G+89B0rX/+q3vfjaPtZOPh4zDI3nntS6bqzFo+EiYiIQsJGmIiIKCRshImIiEJSpsaE0/qtc+NGGVlW2ktvnuDGTZDaSw3SO7Vz49eP9Z7Ckqv2w+KXPeyd0l8lN3VPbyIj9x8Hu/GTjV8IzLc84rE9ad/9HD0jJeSz9h9Y05eM7+XGy7Y0dONdL9l3qorVqqO8p1z949CZVtpHjZ72TXnjwN1nnmvlq4nf4pp3eZRn73KRD+/zn/3cfm5cCz/a+bZtc+OGD3n75nf6HmzlO6/qJ96E2peSrc71xvx1Z27sC10C8EiYiIgoJGyEiYiIQlKqu6PT69a1poe0/TQwb5PhxXe3m/lX1HDjblneJRlPbeho5asyhl3QxWn1wZn7zgSg7yfXWdNtwHqKV70nKlnT3z7vXVvSq5J9o/2Xmn3rxmnwLm3Kf1gRD6sMBJfx1hbvErTat8X2wHnaW9UzVgamberjdTnXejm28v7b/KOIV4KPGSf83N6N226YGtsMSggeCRMREYWEjTAREVFISnV3tFS2b5vSp/ImNz7kp4FWWgPMK5ZlAoA6OX9Hff2Nxd3sfFgYNR+lRoUDNwSmzdvl3bWn/ePrrLTifJhAWZPxjX13useOPMaN7z4ix0pbfrzXZfx732fdeGqufdetC768PKZ5t3nVO0v203dHBOZ7cG4fN248a05gPircljEN7Rc6eeGgjt6QzvcHH2JlW3ug95APPdnbd3bOtLuV5+32ri7p5HuYAwC8f+ITbvyfwy71EibP3veCh4xHwkRERCFhI0xERBQSNsJEREQhKdVjwvl/b7Sm7157kBuf32qalfZ9w1ZunOwna2Q0b2pN/3DA274p73fOjsl1It7JMeFU23myN/407WD/g8DTrXwLdtdz47yFf6R6scqtPatWu3Hl91ZbaW3f8+J/XH4QgrRFbJegpO3vXbbiv1wJAO5Z19mNm1/rnUsScbM0KoIGHy22phfeusuNb6o9143/84F9fk7Q5WPn/HGSNb3jGu+S1NPeGm+lXVTtTzf+4xpvn9tq8j4WugTgkTAREVFI2AgTERGFpHR3R2/ZYk1/ucLrfppwwJtW2spPqntpzx1e5Hlt7Gh3mWTneF1YhzVaYi9XwH12JL4b/1ACdtTxup0zJT0w383TT3fjFij5lzXQvi0b6tV3ZJfnl/d6D5nP/rMU9FmWApHDfJfd5N157uX/e9iN22ZWsd/oexhD6y+9y4vaXzXfypa/zevSvv+bvlbaJf28oaYHunnjGi92sbu082cV36WqseKRMBERUUjYCBMREYWEjTAREVFISvWYcKSad3q3sewx7Dwr7f3OI934gaH2Q6VjMS3XHk/M8/1+6VZhV0RuQTTNnvjFmuYTWlIvt9/GqK/7b1MJAE1ejO0JS1RyrbvMPtdj9mFPufGSPTustEprI7dZSrbsd71bVV6EG9z477PtbW/npiw37nCTd3lg3rZtCNLulrnW9LFtvHM6vuo0xo2HDrWPMxufjhKHR8JEREQhYSNMREQUkjLVHY2pXndv9X/YSQN6XuPGG9tkoahqvxDchb3ivU7W9PRDR0bNF3lJFSVfettW1vS0g1/3p7rR51s7W/kyv7af9kOlz/beWwPTzpw52Jqu9+2MVC8O+fi7prPfDc4X6xPLIvelm9/3bc++3fED+4+x8j3dsKcbJ/vOifHikTAREVFI2AgTERGFpGx1RxcifbzX/VR7fHLL3rGkqv3CodHzafcDrGn5YWZyF4Swulc9azroLllPftvbmm6DKVHzUenxXNfXrOmVed5ZuLUfrVzci0PFqO5z3kM9Dj3xfDee0tW+c+K1N+a4cat/szuaiIioXGMjTEREFBI2wkRERCEpN2PCKRVxg6y0gN82HANOvZ21ot+tDACm53p3SerwwHIrjQ9zL52W33qEG3fPsi87mpzrjQOn85Kksi3fu7ip9kNeva97zb5T2rxzvbuo9X1zoJWm0+ekaOEKxyNhIiKikLARJiIiCgm7o5PBfl448vlohtDUO2ZFYNpHmw9047y164pjcSjF+p83zo3zIzbES6YNcuPmsB+ekl67ljdRr7Yb5s37LbkLSMUu7buf3bjnKzdZaXMv9rqjt9xrd1VXO8u71LQ4727II2EiIqKQsBEmIiIKCRthIiKikHBMOAnyKwaPAa/Nyy3GJSmfJMt7KtapjWYF5lu/K9uNNZf1Utbl53nHGGuuOsJKO2nwBDf+YFFDNy6JD32n+LV+/k9r+rWzGrjx9/uNttJO6HKxG6dNLL7LSXkkTEREFBI2wkRERCFhd3QSvH7Cs9b0vF1e9/R5I29242aYVGzLVK7keXfLeX7ekVbSdUcscePxf7Z248YI5+44VHzmHf2yG+cfbV++1Ol7r+ux9bBtbhzrQ+WpdNjzp31nvHdO6+HGA74eZaWtu2mnG9ebmNrl8uORMBERUUjYCBMREYWE3dFJcNfiU6zpbU83duNmY9gFnWq6x3v8Qs4t26y0DvcNcGOZWRVUtnxxu9e9OPfWhlbaj1Pau3H7x/6y0lqtWuDGeTt3gsoH/x3Rzll0vJX28YEvuvElh13hJUyendJl4pEwERFRSNgIExERhYSNMBERUUg4JpwMx9qnwVfB8oCMlGp5vy+2ppudFdKCULGo+PFUN177sZ3WGpPdeA+IbNtPsy9bmzKpkRtvaFfFjWtORkrxSJiIiCgkbISJiIhCwu5oIiIqd/LWrbemn2/b0o1r4sdiWw4eCRMREYWEjTAREVFI2AgTERGFhI0wERFRSNgIExERhYSNMBERUUhEVfedi4iIiJKOR8JEREQhYSNMREQUEjbCREREISn1jbCIjBSRXSKyJMb8bUVkq4jkicjggDw9RSTfyXdCUhc4yWJZn5JORIaJyG5nPars+x2AiPzh1PvrheRREdkmIvcmb2mLn4h8IyI7RWRi2MsSi/K+TRZGRLKcddgtIveEvTzxKO/ba7LrsEQ0wiIy3tnJbHX+FhSxiAdVNcdXXsFOYKvvLx0AVHWhqmYDmLCPMv9S1WxVHeuUKSJyu4gsE5HNIvK2iFTzzbOxiHwoIn+LyHIRuXwf63y1iCx2ypomIkf60s4XkZVOek/f661EZFLBuhRxfVJKRDo4jcUmEfldRE4rYhGjnM97m1NeDRF5RUTWOH/D/JlVtRWA4TGU20VVb/ct5/MissDZoQ+Ksh7Xi8gqZz1GiEiWLy1HRL4Vke0iMl9EjguaqbOhjnDqd5WI3OBLayoik53vykMR7xsrIt0i1vUYAIV+n5JNRGqJyPvOTnGpiJxfxCIit8nAzyOBbbKhiHwkIn85O/Acf+bC5umkH+vU43anXpsHzbiwunfKWexss+f4Xq8hIjNEpKpvXXOddX1jH+uaciJyrojMc+r4DxE5qghvt7ZXp7yDROR7Z3+7WkSuLUhLYHvtKyK/OmVOEpGOvrQsEXnEqf8NIvK0iGQWsr6FlRVaHZaIRthxlVOp2araLgnlPegrL1tV8xIsbyCAAQC6A2gEoBKAJ3zprwNYDKA+gJMADBeRXtEKEpFDAdwP4EwA1QG8BOB9EUkXkQwn7SAAVwN40vfWxwHckIR1SSpnmT8E8AmAWgAuA/C6iLRNoNhHAFQGkAPgEAADROSiBBcVAGYBuALAjMgEEekD4BYAxzrzbQngTl+WtwD8DKA2gNsBjBaRugHzGQagDYDmAHoBuFm8I7hbAbwCoAWAfgWNrrPxL1LVafGvXtI8BWAXzPe5P4BnRKRTAuUNQ/DnEa98AGMBnFHUeYpIHQDvAbgD5js7DcCoQuZVWN0/CqAvgBNgPqeCH8n3AbhfVbfEs3KpJCK9ATwA4CIAVQEcDWBRAuXVgamL52A+o9YAvkxwGdvANHSXA6gB4GMAHzn7G8Bsq90AdAbQFmafOSTOskKrw5LUCJd0fQG8pKp/qupWmC/wOSJSWUSyAfQEcK+q7lbVWQBGA7g4oKwcAHNUdbqaa8ReBVAHQD2YL/AKVV0J4GuYhgAicqbzeoofMR2X9jA/TB5R1TxV/QbADzA/WuLVF+aH1HZVXQLzQyXo84yZqj6lquMA7IySfCFMHc9R1Q0A7gYwCDBdpjAb+VBV3aGqYwD8guAGYCCAu1V1g6rOA/BCQVkwje83qroJwE8AWorpVbkFwG2JrmOixHQxngHgDlXdqqoTAXyExOqzsM8jLqq6WlWfhvkMizrP02G2wXdVdSdMg91FRNpHFhJD3VdR1V+d7X4XgNoicgiAFqr6TiLrmEJ3ArhLVSerar6qrlDVFQmUdwOAL1T1DedIcYvzmSeiD4AJqjpRVffA7HMbA+jhpPcF8Liq/q2qa2EOUoL2EfsqK7Q6LEmN8H0isk5EfhC7C7aZiGwUkWZFLO8Kp7tvuogE7SiLQpw//3QWzC9t8b3mT+8cUNbnANJF5FDnF9fFAGYCWAVgLcwXoAmA3gDmOI38EJgjqJJIAl5z19+pwyOj5Iu13MI+z2TpBHOkXGAWgPoiUttJWxTxi3iW87pFRGrC/CiJLKsg768AeotIDZhf8nNhGvxHVXVjktYlEW0B5KnqQt9r7vIXdZuM4fNIuhjmadW10636R8Ay7avu14hIFxHpAnN0vgHmyOqaJKxK0jn7nG4A6ooZOlouIk+KSCVfnqJur4cB+Nvp5l0jIh/Hsc/ea1Gx9z7Avx+Ilt5ERKrHUVZodVhSGuH/wBzxNQbwPICPRaQVAKjqMlWtoarLilDe4zCNYz2Y7qaRItI9wWX8HMBgZ2yourPMAFDZ2Th/AHCHiFQUkYNgfiVXDihrC4AxACYCyAUwFMBlauQD+BfMkfSNAC4FcBdM1/d+zrjUFyKS6gapKOYDWAPgJhHJFJHjYX5huuvv1GFRTiwaC+AWEakqIq1hfqgEfZ7Jkg1gk2+6IK4aJa0gvSr2lh3x/si89wE4CsB3MN2+mQD2h/nev+mMq10V70okQaHrGsc2ua/PIxX2Nc+i1mdheS8H8BjMvmsAzPY7DkBFZ1v9VkR6oOSoD/OdOxPme3gAgAPh68qNY3ttAtOTdC2AZjBDc28luJxfAegh5qS8CjC9RBXg7Qc+B3CtiNQVkQbwGsxo+4l9lRVaHZaIRlhVpzjdF7mq+gpMg/aPBMqboarrVXWPqn4GMxZwelB+sU/gCvr1NgLmSzUewBwA3zqvL3f+94fpZvwTwDPOPJcjusEwjUonmC/CBQA+EZFGzvKPU9XDVLUHzK+ybgBGAngNpjvtbgAvFvYZFCdV3Q2gH8xY+CoA/wbwDoLXPxbXANgB4DeY8ea3CitPRD731WH/OOe5FUA133RBvCVKWkF6tLGirRHvt/I63WfnqGoXmA3/CZjx/1tgjpKPA3C5/8SRYlaUdY21vIIyYiovxm0ykXkWtT4D86rqTFXtqaqHwvRqXAxzEtKLMN2+FwF4TUSi9RiFYYfz/wlVXamq6wA8jAT2uU6Z76vqT073/p0Ajgg4Ko1pe1XV+TAN+5MAVsIM2c2Ftx+4F2acfiaASQA+ALAb5oCgSGWFWYclohGOQhG9izMl5UWcwBX1170zbjJUVXNUtQlMQ7zC+YOqLlXVk1W1rlORtQFMDZhlFwAfqzkrNF/N2Z4rARzhz+RU+JMwDVIdAOmquhRmDGz/2Fc/9VR1tqr2UNXaqtoHpmcjaP1jKe9vVe2vqg1UtRPMdzWwPFU90VeH8Z61OAembgp0AbBaVdc7aS39Z0k66XOiLMsGmPqMLGuvvDAnsU1W1V8B7AdgmqrughlzDKu3YyGADOdklgJBy79PRfw8Ct6zz20ywXlade2Mg7cKWKaY6x7mhMIhqroDXn0ugTnyDDqJr1g5n81ymP1issyOKK8gjrrfjXV7VdXRqtpZVWvD9Bg2h3MOgDM+f5WqNlbVlgDWA5iuASeuFlZWhGKtw9AbYTGngPdxunEznF9FRwP4IoEyzxSRbBFJc7pGL4A5sSSR5awl5hIhcY5QHoY5sSHfSe/gdJ1WEJELABzv5InmJwAniUhLp7zeMONwv0bkGwzgZ1WdCfMFq+TMuxcSOJMxFURkf6cOK4vIjQAawhy9x1teKxGpLeaM8RNhGquEr8lz6qcizM4h01nmgu3gVQCXiEhHZ0xxCJx1cMZHZwIY6rznNJgfQmMCZvUqgCEiUlPMyT6XIuLzEJF6AK6EOSkIMF14vcScA9ANIdWxMz76HoC7RKSKM5RzKkxPTLz2+XnEw6nLgsvIspzpWOb5PoDOInKG857/ApjtHDFZYq17ZzuuqKqfOC8tBnCMmLPKs2C24ZLiZQBXi0g957t+HczVDYmUd5qIHCDmMqE7AExM9BwHEenq7APqwpx5/XFBHYm5LLSRsw89zJnn0HjK8uUp/jpU1VD/YH5Z/ATTtbMRwGQAvX3pzWC6g5oFvH8kgHsiXpsAM2azGeYEinOjvG88gMEBZfYEsDzitbYAFgDYDmApzKVC/vTrYE6q2gYz1tstIn0rgKOcWGDGeZc56z0PwICI/HVgGuVqvtf6w3T3LgHQK9b1KaZ6/B/MyQxbYcZqWgetf5T3DgPwesRrZwP4y/m8ZwLoE8v7ItI1ynKMd173//X0pd8AYLXz3XkZQJYvLcd5/w7nu3BcRN3M8U1nwQxhbHbKuyHK8r0K4CzfdFMAU5zP8aGIvINgdmrFVZ+1YLr3tjnf0/N9afFsk7F8HoHfYUTZJn11bP3FOk+Ybv/5Tn2OB5DjS3sWwLOx1L1vXjMBNPe9dizMtroSEfugaJ9Rcf7BHNU9DbPPXQVzHk1FX3qRtlfn9X/B9AxugLkEqGks74uoy8jtdSLMPvJvmIazii/taOfz3e7USf+I934O4LZYygqzDkP5AiT5y/SC84X5I8b8bZwv3nYAgwLyHO1sbBsRZedfkv5iWZ+S/gdzxLnNWY8qMb5ngVPvIwrJsxPmx9jdYa9jgp/PV87OY1zYyxLj8pbrbXIf65rlrMM2mEueQl+mONahXG+vya5DPsqQiIgoJKGPCRMREZVXbISJiIhCwkaYiIgoJBn7zpI8vdPO4gB0SL7KfzfpF5mzPsOTivoEWKdh4jZatsRanzwSJiIiCgkbYSIiopCwESYiIgoJG2EiIqKQsBEmIiIKCRthIiKikLARJiIiCgkbYSIiopAU6806iIio/EmrXNmNu07aYqUNrTvTjY+fe7obV+i9NPULVgLwSJiIiCgkbISJiIhCwkaYiIgoJBwTToGMBvXdeFebRjG9J3PhCmt6wa0t3bjGXO8+4LXm7bTypU34OZ5FJCo1dvY9xJqu9PkMN9ZuHd148SlVrHxHHfOLG0/4Zr/A8hv+mOfGFT+eGvdyks0/Drzw+XZu/EHd5618+b74z1kN3bgVOCZMREREKcRGmIiIKCTsjo7TpgsOc+P1/7C7iG85cKwbD6z2WUzlvbSpmTV9etX33bjmWRUD33dy464xlU9U0qXXqe3GeaMqufHbbR628q3Oy3Tj6mnj3bhZRmUEuvD7wKQ1F2x3478er2Cl/XP4tW5c+4Ufg8unvSy6vYsbz+31uBv3X3SilW/9vS3cuNXYyalfsBKGR8JEREQhYSNMREQUEnZHR0jr0sGN51/tnW054fhHrXx103/y3pOE3zKXVF8W8UpwFzRRWbTwMW9IZkH7l3wpdjdzvXQvfnpjWzeescUe0lm+rUbgvNLFOyf303YfRy0bAEYN+Z8bXz7vKistbeJMULBd9fZEfX32hDbWdIux5bubn0fCREREIWEjTEREFBI2wkRERCHhmHCEbS2quvHCE5/xpVTaO3OCnt3o3RXrjaUHx1VGdfyerMUp89IO8O6utLOBfXelJf28u5KdechPVtpu9QYKv33Nu3tTw+82Wfn05zlJWc7yQg/vYk2POuI535S3axq7wx4Tvv+mC9246px1XsLav618aRv+DJ53mlenbR+6wo3nnv2Ela9VZrYb7xiy2UqrPsi7M96eVasD51VeZWbvcuMt+V7c7KvcMBanxOKRMBERUUjYCBMREYWkzHZHZzRpbE3P+08TN64/yet6rPaWfYeWtFx144W7vS6UP/fYlzs0zdjoxoN+vdBK2zDPu/NP/Z+88mpMsrvHdOtWN66+kd3KyaDdD7CmF13pxW8e/oIbd60QcS1KrG7ybvC/48ZdVtLzG73u7qdn9bDS2lwyz43zd9p3WCuvdle37051QAVvd5QPb7u56eWLrXxN35/kxnmIU773ztbXe/uADhXsy5Bmn/qYG3+332grrftxXjd29dfZHZ3euoU1PefoEW587V/Hevm+nQHy8EiYiIgoJGyEiYiIQsJGmIiIKCRlakw4vUZ1Nz7k08VW2gd1PnLj7tPscR+/rM+9y1NuOmmQG+fNWWDPq4N367VaC/6w0mrlL4xadvSbuFE88o/0xn6XeENz+LT7U1a+Vhn+S8u8ceCvdtiXnN02t58bb1xmj///2s+7bOWO1d7Tsx5sMM3K16WS9xDyhw8ZZaXdev0gN25y3yQQkFdRAtP2nzTIjZvdW3yfV5srp1jTnxznPWT+rOz1VtrGU7a5cfXXU7tcpcGCYcG3CS1OuSd6l3tuaRrcxNWdbl9yptPDucSQR8JEREQhYSNMREQUklLdHZ1W0X7SUO5orzv6tjrfWGnt3vP6LNu/73U7FHaJQ2QXtJU277cYl5KSYdGb9qVHbwRebmR3M5+3uLcb/zTfu4Si/bXzrHx1t3l1XTdi3pd3Pc6N11zT3I2vf8a+zGlI/fFuPGFHQytt5lVel3a/10914z1/Lkd51e7W4O6/9OlVA9OK0+0/ecMUZ/V6yUq7stP3bvwJahbbMpVUjxw6KjDthzcPcuMGSHx44Y83DrSmHzv0LTfer8JEN66fnhVYxu+77QHCU0df78atbpwcmT1leCRMREQUEjbCREREISl13dHpNb1un/l3t7XSFnR42o2nR9wjvP1di9w4b7N9VhyVDGlV7Icq/HbXfm48r4d91nOa70znn3x3Oev/4ZVWvnZ3et3ObTd6ZzPnI3b7VV3hxl9leF3a0/7X1cpX+2HvzNp+VTbCFnwmcHmStn97N+5Z4ysrbeFu705idWbvLrZlKkzN73xDXr3CW46SKr1aNTeukmbvdL/c4W3PDR6JrQtaMnlgQPYAACAASURBVL27qO3qtb+VdvszL7vx0RWnW2mZ4u0PpuZ6XdAD559l5buhxZdufEqV7Vba0/284YZHR5zmxnlzo1/tkiw8EiYiIgoJG2EiIqKQsBEmIiIKSakbE/7rgg5uvOA0+wHcH23zxotfOrm3lZa31r6rFZU8G0/Zz5r+5qz/c+M02A92H7fDG/e5/wrvKVatv7QvLYj1KTuS4W0Kae1aWWkvflDLjf/36ituvF+FNRGleMuYLvbv2/2mnO/GjdeU3+/ibxd6d1U6N3utlXbk7AFuXO2zn0Al3+LrOrvxkRXHWWkdvx3oxq3xc2AZ/qcvLbiyvhvPPfuJaNkBAON2ZFvTV3wxyI3bP7bOjbMW2tvaU/DOI3piXFMr7ZP277nxfc28y10rzA1cjKTgkTAREVFI2AgTERGFpNR1R285dEdg2mOLvQdHV1pYfrv8Siu1b0CFnRp8Wc+WfO/OWKsO9S5r2HH6IVa+1m1WRn3/pp323dbOau49aPzKGq9ZadN2eeV3z/Jf3GR3kfv9sNO+CKrxPd66aG5uZPZy4/oTP3Vj/yVJAFDhqdq+KW6/pYHsH3y5Z+YflQLT/PwPfpjfy7sUMfIywv6LTnTjzTc3ttLa/OhdHhjrENTvixrYL7SPni/VeCRMREQUEjbCREREISl13dFvdX/eN2X/hhjd0Xuo5+EP/9tKa/HRLjdOHz8DVPLU/NC+of9lA/u78evt7Qe2nlLFu0vWGf/y7pSWp8H3wspV74btWVLYV99Os7ugPXsiOr56zj7XjWtdaafponCeVVqSPbf+aGu64idTQ1oSilf7equL/B7p2smafv/IZ3xTmW7UafxlVr42l3h3v5Ods4o833357xrvOcQVx//ixkW5u148eCRMREQUEjbCREREIWEjTEREFJJSNyZ8SJY3ZrBb7XG3mmneZSfzz7GfurP7bC9v53GXu3H1n+xLVbY28cYaq3kPXkKd2dsCl2nd/vbTf+qP9+6klMdLpWKWv2WLNZ11vDd9Wf3TrbR5w3Lc+Piu3vjNwk31rHxLV9Rx4/QK3nfglHazrXwPNpiGour4rT1m1e7f3tOW9qyOvJtW+ZReo7o1XTVteUhLQqnQpLL3tLC0yGM6UUSz8Josa7pDprdP7/rTBW7cqr99l61kj81mZu+yprft8ZYrf+fOyOwpwyNhIiKikLARJiIiCkmp645u8fGlbrzw5Gdjfp//oc8LjnvBSzguKYtlmXqLd3ek6+b6Lls5ObUPhy7L8iK6d9v+y5te4nu9ApZa+dpETBf48v2O1nRh3dFL9ngP/+73xM1e2Y/al9Tk7dkDsi2/xL4cpX/Vb914xracYl6aosv9x6bAtO35FQLTyot89Y7j8iM7jAPueNew/kZr2v++jnW9S542JGH5IvkfFjHn6BFW2tGzz3bjasV4xzYeCRMREYWEjTAREVFI2AgTERGFpNSNCbe70jttvc+79iUiA5/82I0rp9lPqjm5svcAcf/4cCockuWdmj/xwDfcuNP/rrHytbrpx5QuB9kWDz/cjWcc/EhEavD43pkPeuPAjZ6a5MbRL8Cg0mzPMV2t6bcPfNI3ZV9a8/4D3lPbqmNyKherTKlxiX35z5QJ3iVKTzbz9uGHP3Cjla/t4975HXtW/BXXvDuM8spYnWc/ka/iY7V8UxwTJiIiKvPYCBMREYWk1HVHq+8ykMyvp1tpb7VvFPi+x8/0LhXKy/ROnT/iRvsyk/sb/JToIlr8d5Fp0iX6A+Ypdf666Qg3/qL/g25cSSoHvuexDa2t6QYvz3TjVD9RhYqfvwv672vtO+O1z/S6oK9Y0d1KqzHKexpbeRma8F/iAwBHV/+myGVEdiU/cFw/N+4yxrtN4a8XPG7lu6JHLzdeeVItKy1v/d9uvHGAN+x05HVTrHz/rf+DG3d92+7ubjU2nCEFHgkTERGFhI0wERFRSEpdd3S8qoyeEvX1j7scbk3fP8Drjt6u3g2+u37/Lytf8xe9M6zXXbPdSpt2sP0Aeio+u4/vZk1/cJXXBd0sI7gLepnvrlgf/edYKy1re3KHKMqTakvsh6z47z4WJsnwdn0br/ceFDLtoLetfF/tqOTGC++w7/5VYXfRH/pR2uX9vtiafnvVIW58WquxVlrzI5e5cXq1al4Zmzdb+fYsWuLG0w/0jguPHmBfTVJrtnenLamz20pb/GRTN55ztHdGe+QZ0P4u6FY3lowz2nkkTEREFBI2wkRERCFhI0xERBSScjMmHKTZF/adtTDACyuLdxeleT1esrM17+3Gn+V8EVFq9N82y1bZp9W3sZ7/Q8mw5GT7bmg5AePAK/PsscmB1/3bjSt/Gv38ASq6KmPsz3Ls3R3cuFXFtVbab006u/Ge5SsSnnf+kQe48eIr7LQzOniXnQ2vZ48D+w2/8UI3rvTF1MB85dXOwd5Y78Nj2ltpn7T/0I2vHedd3jX1Wfs8nOy/oj99bO3B9gWBB1/jXb70UKOJVpr/UtDnN+W48cj/O9nK12pEybtLIY+EiYiIQsJGmIiIKCTlvjs6c9pv1vRhM85z48kHvRX4vtdyvvJN2b9lctU7ff7kud6dutpfY98U3L54g+KVXtvr5v/59EcjUrMQTc+JV1nTrd5nF3Rxu6KGfbnL6k+8rs1pfzdLuPz7WzzvxgdUCN7VTd/lbYkDpl5ipbX6Zr4bc3vdW95Cb5/2/an2JVw1P/XuPvZIowlewl0TEMTfrZxfhPvTdZ54kRu3vmGdG9daUfK6nyPxSJiIiCgkbISJiIhCwkaYiIgoJOV+TDh/yxZrusHVNd2474hT3Pi2nE+tfIdneSNEY7bWsdJu/+wcN259vXdrNI4pJU96Ta+erpvijTFlS/QxYAB4YL13eUybS+1zAfh0pOLhv2RkzbXfW2l31p3lTfjjuHm7tz0RW98s7460uGCUd3vEFrfYY4jcZmPnv/0kAHzQ07vk7PGLvCclbWth33LyixO88zj6fHGdl1DIo6navbjTms75aba3HLEsbAnCI2EiIqKQsBEmIiIKSbnvjo60Z4n35A8c44XXXGPfcmfLwd7TOdoPWWeltV5aMp7OUZatO8W7O8/xlb9147xCurA+u7OnG1fZxkuSwlDLd8ein75va6U9/IHXxXhDTXu4IB7tv7vYjSv8Yt85rcl9k9y4BUr+ZSylUd7qNW7c+P41gfmuhnc3rbaI7YllhWzmpQ6PhImIiELCRpiIiCgk7I6OUf3HJ9nTvri0nY1XFpxx49dunKfB5za3/vhyN247hl3QJUnkA+K/7lzVi3FQwuW3xMx9ZyIKGY+EiYiIQsJGmIiIKCRshImIiELCMWEqlbpU8i4lSxfvt+TknfY9jjo+6F0awbF7IippeCRMREQUEjbCREREIWF3NJVK173hPXx9/qVPu/HFI6628jVdZF9aRkRUkvBImIiIKCRshImIiELCRpiIiCgkHBOmUqn5UG+st8/QA9y4KTgGTESlB4+EiYiIQsJGmIiIKCSiWpYej0xERFR68EiYiIgoJGyEiYiIQsJGmIiIKCRshImIiEJS6hphERkmIrtFZKuIVInxPX+IyC4Reb2QPCoi20Tk3uQtbfETkW9EZKeITAx7WWIlIiOd+lkSY/62Tv3nicjggDw9RSTfyXdCUhe4GIlIlrMOu0XknrCXJxbcRgtXGrdRv/K+vYrIcc5y5ovIcYmWF0ojLCIdnC/iJhH5XUROK2IRo1Q1W1W3OeXVEJFXRGSN8zfMn1lVWwEYHkO5XVT1dt9yPi8iC5wPe1CU9bheRFY56zFCRLJ8aTki8q2IbBeR+YVVlrOjHSEim53ybvClNRWRySLyt4g8FPG+sSLSLWJdjwFweQzrmlQiUktE3nd2kktF5PwiFvGgqub4ygv8TFR1oapmA5iwjzL/cr4nY50yG4rIRyLyl7NDz/FnLmyeTvqxTl1ud+q2edCMC6t/p5zFIrJSRM7xvV5DRGaISFXfuuY66/rGPtY1qUTkKhGZJiK5IjIyjiIit9FezuexKdrOm9to8RKR8c4Pga3O34IiFhG5vRY0zFt9f+lAQturiMjtIrLM+dzfFpFqvnk2FpEPnc99uYgEfqbO9+8XEdkoIuudfVVjX/pNIrJORH4Vkc6+17uLyAf+slT1a2d9liEJir0RFpEMAB8C+ARALQCXAXhdRNomUOwjACoDyAFwCIABInJRgosKALMAXAFgRmSCiPQBcAuAY535tgRwpy/LWwB+BlAbwO0ARotI3YD5DAPQBkBzAL0A3Czer8FbAbwCoAWAfgUbtLPzXqSq0+JfvaR6CsAuAPUB9AfwjIh0SqC8YQj+TOKVD2AsgDOKOk8RqQPgPQB3wHxvpwEYVci8Cqv/RwH0BXACzOeU7rx+H4D7VXVLPCuXZH8BuAfAiCSVt80p66YklVeA22j8rnIavWxVbZeE8h70lZetqnkJljcQwAAA3QE0AlAJwBO+9Nfx/+3dd5gURd4H8O8PFhYWkLAnoMICgkRJAioqQcyeotzpKcopJsTjzHJ4ioKgr97r66lgVjCdkdMVuedMJ3BGRJAkSSUJBkAkS1r29/7RvdVd48wwuZbd7+d59nl+NVVTUz29PdVd1QFYAe8357cA/kdEjo9R1yIAp6hqPb+urwE8Cng75wAug/f/8RiAe/zX8wDcB+C6NJcjLhdHwm3hfQn3q+peVZ0K4GN4X3aqzoT3D/CLqq4EMAHApek2VFUfVtX3AeyMkn0xgAmqulBVNwIYC2Aw4A2/ADgCwChV3aGqrwFYgNg//hcBGKuqG1V1MYAny+qCt2FPVdXNAD4HcKi/N3gzgFvSXcZMEG/I8fcAblPVbar6EYA3kd46jfedpERV16rqI/C+x2Q/83cAFqrqJFXdCe9HubOItI2sJIH1X0tVv1TVefB2XApF5EgALVT11XSWMVNU9XVVfQPAhgzVN1NVnwewPBP1herlNlpxnQlv/a1W1W0A/gbgPBEpEJHaAPoCuEtV9/jb0j8R43ff3/a/D720F0ArPy4CMEdVtwD4D7zOGPA63zf9PiVrXHTCEuO18BDAJhE5Lo16rfqypAO8vfAy8wA0EpFCP295xBHNPP91i4jUh7dTEllXWdkvAZwkIvUAdIe3RzcWwAOquilDy5Ku1gD2qupXodfMMohIkb9OixKpLIHvJOMS+ExrffvDrMtitGlf63+diHQWkc7wjs43wjs6viYDi5ITKW6jucZtNL67/SHYj0Wkb9mLyW6vIX/yh4Zni0isnZlkCH79u54Pb0RCQq+F82P+7pctF4AdAG4C8L9+1jcAOvrr70QAC0WkKYDzAfxfBpYjLhed8BIA6wAMF5FqInIygD7whpMBAKpazz+aStTbAG4WkToi0gre3lDBPt6TrtoANofSZXGdKHll+XXwa7Uj3h9Z9m4AvQD8F96QbzUAnQBMEZEXReQDEflzqguRIXGXV1W/9ddponMo+/pOsmFfn5nsOo1XdiiABwE8AW+04CoA7wOoISLv+POUfVJZiFxJYRt1gdtobCPgHfEdAu//cIqItARS2l4BYBy8zrEhvCmbZ0Tk2DTb+BaAy8Wbu6/rtxkACvydp48B3CYiNUTkCHijGDF/98uWC8BvAIyE1xdBVTcAuAvAVHjD2jfB2z5HABggIv/1556bpLk8UeX8KUqqukdEzoY3tj8C3tzaqwB2pVHtNX59X8MbPnsJwMBYhUXkLXgbDQBcqaqpnPSyDcABoXRZvDVKXll+tLm+baH8nZFlVfVnAOf57a4C4AN4P+I3w9sDHwzgCxGZqqqLUliOTEhmeROtr6yOX30n0YjItlCyfRY+M9l1GrOsqs6FN5RWNh91H4Ce8H7Er4M3H/uBiDTTSnpfWW6j2aWqn4WSz4rIQACnw55zTaa+8Jz8v0XkBXhTOB9HK5/g9joRQFMA0+H1VffBG6Je4+dfCG+nZzW8aY4X4tQVbuvPIvIsgHkicoiqlqjqS/D6DYjIb+H1R3MQjHj0h3dUfP6+6k+Wk7OjVXW+qvZR1UJVPQXeHtnMNOr7WVUvVNXGqtoB3nLFrE9VTwudPJDqWacLAXQOpTsDWOvvVS2ENy9UJyJ/YZS2bATwQ5S6flUW3klsM1T1SwAdAcxS1d3w5rKyPfwez1cA8kTksNBrsZZhn5L8TsreEz4hJOmzFhP4TGt9+/PgLWO0KeH1D++kwpGqugPBOl0J72gq1klCFR630ZxTRJ8qzEp9iWyvqlqqqqNUtbmqNoH3fX/n/0FVV6nqGap6oKoeBe8Eu0T7kTx4R+3WjpiI1IR3lv6N8I7sV/tzxZ/DG93IOFeXKHXyhxAKROQmAAcBeCaN+lqKSKGIVBWR0+BtCGlfUyki1UWkBrx/pmp+m8u+s+cAXCYi7f05o5Hwl8GfG50LYJT/ngHwVuBrMT7qOQAjRaS+f6LPFYj4PkSkIYBh8E4IAryzAo/3T1Dojgyf8JIMf370dQBjRKSWPwx1FoDn06h2n99JKvz1WXaZSr6fTuQziwEcLiK/999zO4D5qrok8jMSXf8ichKAGqr6L/+lFQD6iXdWeT4ydFJUKkQkz1/OqgCq+suR8siZiFTx66vmJaWGiFTPQDu5jSZJvEvhTilbpyJyIYDeAN5Jo85zRKS2v55PBjAI3smZ6bSzgf/bLiLSHsDfAYxR1VI/v50/BVldRAYBONkvE62u34lIG799B/rl5vijGGEjATzjn8T1LYA2ItII3hnx2Vl/qprzPwD3wjsZZRu8cf9WEfnbAPSK8d7RAP4R8dof4A3h/QJvwzolkfdF5GuUdkz3Xw//9Q3l3wBgLYAtAJ4GkB/Ka+6/fweApQBODOVdCO9M27J0Pryhly1+fTdEad9zAM4NpZsC+Mz/Hu+LKDsYwEc5XqcNALwB71KUbwFcEMor8tdpUYz3PgPgzojXEvlOpgO4PEadfQGsibGerb9EPxPeSRtL/HU6HUDzUN5jAB5LZP2HPmsugGah104AsBLeUdf5+/qOsrw+R0f5rkaH8pPdRvtGqW/6vt4XZd1xG01/3R4I78huK4BNAGYAOCmUn8r2+iG8efIt8IZwz4/yvulIYnuFd8LnUni/66siv3N4Uzfr4f3mfASge0S++R8FcDW8naLtAH4E8DJC255fpo3/veSFXhsO4Cd4J9t1jCi/EhHbdUrrw8U/QZr/QCP9L3ITvEs9EnnPUn+FTIxTZqf/TzTW9TKm+f28529c77tuSxJtftJfP8sSLH+Yv/5/ATA4Rpne/o/rJkTZKdtf/vwf/03+//wo1+1JsM3cRuMv6363jUa0v1Jvr/B2lsvOsj4+3fr4PGEiIiJH9rt7RxMREVUU7ISJiIgcyel1widVOZdj3468Vzopk5cfAOD6dCkb6xPgOnWJ22jFkuj65JEwERGRI+yEiYiIHGEnTERE5Ag7YSIiIkfYCRMRETnCTpiIiMgRdsJERESOsBMmIiJyhJ0wERGRI+yEiYiIHGEnTERE5Ag7YSIiIkfYCRMRETmS06coERFl0jf3H23iZec9ZuVdtKq3idf23JKzNlFySvp1M/GKAUGXdOMJ/7bKDam70sRVYD+gqBTBw6JGretq4ikrD7fKHXx31SAxc0FK7c00HgkTERE5wk6YiIjIEQ5HU4WW17iRiTcf29zE351kP+t8Rf8nTLxH91p5x84938TrV9c3cft7frTKlaz8Nq22UvKOPXpRzLznmn1g4l4DrrTyCoo/y1qbKqvvRhxjpbcfttvEA7vNjPm+OxoG214pSk1cJeIYMZzXbvoQK6/hm/kmrvPKDBMfjNj/H+UFj4SJiIgcYSdMRETkCIejab8n+cFQ1PI7jrDyHjrnKRP3qflLzDr2aLA/Gh72AoAPu7wYJLqEwsJLrXJF5ybUXMqg8JBzPN/3ts+mbVWcjdZUbvOuechKh89YXrt3h4kf2WAPW7d+K5gqqPV1dRPX+MmeMiqc8KmJW2JOeo0tR3gkTERE5Ag7YSIiIkfYCRMRETnCOeEIe/sGc4p5t6818ZQ2b1rlqklw55V4l7QU3lrNxLLyO6vchjPbm7jBG19aeaVbtybT7Ert2+HBHXcW/PHBlOq4ZNUJJp7Q7L2E3jP3mIlWuj96pPTZlH2trp+x70KUlt4LzrHSUzu+YuLwPPDsrvaxX2vMym7DyjkeCRMRETnCTpiIiMiRSjkcHb6kZWv/LlbeqLuDIcbwJS32RSvAntDZ8/EuaTnitsEm7tzY3ueZ3Dw4pb9HvautvEbjP4neeAIAaM/OJp546fik39/p6WusdIuxX5i47f3DrLwlZz2cdP1ElU29K3Zb6X+9X2jis+vNNvHcdhdY5fYu/jq7DSvneCRMRETkCDthIiIiR9gJExEROVIp54R39e1o4qkPPBSz3LQdtU18+532LQqr/aKRxY0tzYJ9m+qhOyX+5Sb7kpbNpSUmrv2DfZkT2cJzwACgd/5s4m7BFP+v5u6LtzU08cTB/U3c/DP7qS5aGnz/ba6fZ+Wd9sZVJh77WPDEl+759jo78cvgsrL/HF4nchEoC1q+MtTEy857LGa5b+4/2krzkqXMK1m9xkrfXHyhiRcNCn5ndze2t42qi7PbrvKOR8JERESOsBMmIiJypNIMR4eHM+9+9PGY5QYuO93EW0Y1NXH9aZ9GKx5V3VYtTNxl0jITt6tu7/O0nXy9iVv/kw8Zj2ddj1pW+vO2wdB++O5lm0vtyyRGvRrcvaz5p4mtQ921y0pXeze4o8+gd4Lhz4Vn2lMZwxsE6/rJly628loMtIe4KTPiDUGTY6EHV1UJJTZ0qGEVayDdkIj8WcGlTHu3bEmvbeUIj4SJiIgcYSdMRETkSKUZjt54a/BQ6fDZtKcv+Z1VrupNBwTxnC+Qik3dGpl4VMNXY5Zr+m5K1VdKVU7cYKXDdykL373skuX9rXLNb0t8GiERra8Kzqoef1wHK++GBktMfGH7z628T1AdRBVZXtMmVvqes18wcSmCjXTGX+2HrFQJHQuGt+sqEceIfReca+Jdk+xtr3BCZrfzXOKRMBERkSPshImIiBxhJ0xERORIhZ0TXvFyJyu9sOvTJl5TEswPV7m1vlVO58xP+rPCT2UCgFbXLQrqD+3nhB8cDwA137Dv2kS2vEMONvGNbf6T0HuWTzrMSjfC+oy2KWzi5BOt9A2XLIlRkqhiCs8Dn/6OfRle/1obTTxqXVcTT1l5uFVOZ9SLWnf/8z+y0jccGvwGnD1mk5VXOiaYcz71j0NMHL6sCSiflzbxSJiIiMgRdsJERESOVNjh6Iva20O94VPfV5UElyFhRvLDz4A9BL30AfvhApOLgofAhx8osOreNla5AvAuWfFsPK7IxOfUnhyz3JDVfU18SOgOZQBQAjcOr2nfzH7mof1MXLJ8ZY5bQ5Qd27oEU0ZD6trbaO/5fzDxAacF2+XBWIREzP6bfYw4r0kvE4+8vJmVd/SpC0z89vPBQ1Ye3tTSKvfWJUEdmLkA5QGPhImIiBxhJ0xERORIhR2OzrSqHeyh5MVX1zXxkjMfjixuhJ9JXOeTFVYenyAc3/ojZN+FACy7p52Ja/5YPs44P6OWfYevv3dvbOLaHI7OOT4/ODtqTAm2tzOm2A9iOADLIounpWTNdyYuGv2dlff96CDuOuJqE0eeYT32leDBL3+9bKiVlzd1dgZamTweCRMRETnCTpiIiMgRdsJERESOVNg54ddWdLHSwwuD09G75m83ca/5OxOq78iC16308TWD95VGFg65cd45Jm6ydmFCn0WevQWxn6gSVl7uPFZNqpo4/GQnIsqdQ/72iYnnvdDUyjvonc0mHvPUk1betXcNM3Eun8rEI2EiIiJH2AkTERE5UmGHoxsPsk9h7//GABP/q21wZ5fwMHUyeoVOgy8daF+O8mGXF03c8MmClOonoFOnlSYujTvoXz7s0eCis/2hvUQVXfiyJgCYdMspJv5htH3Z2iMjx5n44qbXmrho9CfIJh4JExEROcJOmIiIyBF2wkRERI5U2Dnh0q1b7RdOCNL9BvzJxOu6xd4Pqb84uM6k7gv2/MH653eZeEmXl628CZubm7hg4Q8mdvVEH8q9VSW7rXTN9btjlCSiXKk5Obiccd7s2Jcvzb3iQRP3H90jq23ikTAREZEj7ISJiIgcqbDD0fEUFH9m4ubFqdWxpN9TJo68HOXhpX1MfPDqxB5gTfufy89+N2beWU8Pt9JF07J7mUNlddGq3iZ+rtkHMct9c//RVppPVaLIy5fGzTvexEP7LM9ZO3gkTERE5Ag7YSIiIkcq5XB0Kqp2aBPxSvAA6MgzYRuNq5GDFlV8228/2MSznq5q5XXPD+5O9e2kjiYuOje1O6ClokfNFVZ65i4xcfN751l5vH8WUTlzZEcr+fzRE0z88KaWOWsGj4SJiIgcYSdMRETkCDthIiIiRzgnnKDlo6rHzDt3zuVWuvG0L7LdnEqhyn/nmHjYA3+28j4fMd7E7x31qIkHH3+NVa5qhtfFipc7mfjYGrOtvGPmDDRxg+1fZfRzKfDLgKNM/Fyzxx22hMJW3XGMla7xUxA3Gl8+LtGr2r61ibeM2W7lNcnbYeK3B/cK5WT3PBMeCRMRETnCTpiIiMgRDkfHoT07m/jNox6JyA0uQ5L36+eoRZXXQdN/ttLd+w0y8awe/zDxmr725WHNpqX/2dt/Hwx/vnpU8ODvT3flW+Ua3MlL03KhxV8Wu24C+TZc1tPECy4fb+W1mx5M0zWys9KW17SJlV51QVHUcoeebt/56pamL5l4xg77MqQBo4O73DX4/NN0m5gwHgkTERE5wk6YiIjIEXbCREREjnBOOI51PWqZuEWePd8XfnJS3k7NWZsqq9L5S6z0IbcGtxEtLm5g4jcH32uVO/U3N5j4sGGfIRbp1sHEa3vWtfIevzF4wHe76sF+3YG+0gAAAslJREFUa9spQ6xyrWfMBGVe+JIkIPHLknoNu9LErYr51KRsqyb2rWUX9w2eNDdnRfB7ecGnV1jlJBT3PvQbEy/d1NAqN63jJBNXgX3pYSk0lBfU+MimFla5gVOD/4n2o3+w8hqsyd08cBiPhImIiBxhJ0xEROQIh6Pj2PmbYIijNOI5OA/83N7EhU+6GcaozPYuXGriZ08NHsb9+BP2enr7jL+b+NVe3Uz88ov9rHJPDQmuoeiaH/uZR6cuOsfEbR/dauXxSUm51/KVoSZudb095FyA2NMPlBmFE4LfvmO2D7Xy1p25K+p7nu05wUofmR/8zoafXlRqDVTblzyVbrDvYHho8Z6on1V99jdWuvWWWSYuifqO3OORMBERkSPshImIiBzhcHQcg86OfbuliZNPNHFzcDjapZLlK02cP/BAK29o12tNXG3EjyaeffWDVrm2U4bFrL/F68FAc/60+SYu3bM76bZS8gqK7WHlU4q7mLgVeNZzeVHn5RkR6ejlxuCIBGu0p3taYk6McrHtTfoduccjYSIiIkfYCRMRETnCTpiIiMgRzgnH8dqKYO5peGF2H+xMmbF3/XorXe3dUPrdIOyPHla51kjsble8NxoRZRKPhImIiBxhJ0xEROQIh6Pj0PeDBwPc0sS+iXyjWfvDye9ERFSe8UiYiIjIEXbCREREjrATJiIicoRzwnE0GveJib8cZ+fVTPCSFiIiolh4JExEROQIO2EiIiJHRJX3ACIiInKBR8JERESOsBMmIiJyhJ0wERGRI+yEiYiIHGEnTERE5Ag7YSIiIkfYCRMRETnCTpiIiMgRdsJERESOsBMmIiJyhJ0wERGRI+yEiYiIHGEnTERE5Ag7YSIiIkfYCRMRETnCTpiIiMgRdsJERESOsBMmIiJyhJ0wERGRI+yEiYiIHGEnTERE5Ag7YSIiIkfYCRMRETny/wgqoaeV1TVBAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x576 with 16 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "with tf.Session() as sess:\n",
    "    ckpt = tf.train.get_checkpoint_state('./')\n",
    "    if ckpt and ckpt.model_checkpoint_path:\n",
    "        saver.restore(sess, ckpt.model_checkpoint_path)\n",
    "        final_pred, acc = sess.run(\n",
    "            [pred, accuracy],\n",
    "            feed_dict={\n",
    "                x: mnist.test.images[:16],\n",
    "                y: mnist.test.labels[:16],\n",
    "                keep_prob: 1.0\n",
    "            })\n",
    "        orders = np.argsort(final_pred)\n",
    "        plt.figure(figsize=(8, 8))\n",
    "        print(acc)\n",
    "        for idx in range(16):\n",
    "            order = orders[idx, :][-1]\n",
    "            prob = final_pred[idx, :][order]\n",
    "            plt.subplot(4, 4, idx + 1)\n",
    "            plt.axis('off')\n",
    "            plt.title('{}: [{}]-[{:.1f}%]'.format(\n",
    "                np.argmax(mnist.test.labels[idx]), order, prob * 100))\n",
    "            plt.imshow(mnist.test.images[idx].reshape((28, 28)))\n",
    "\n",
    "    else:\n",
    "        pass"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 全连接神经网络：\n",
    "不适合做图像识别原因：\n",
    "\n",
    "    参数数量太多 \n",
    "\n",
    " 考虑一个输入1000*1000像素的图片，输入层有1000*1000=100万节点。假设第一个隐藏层有100个节点，那么仅这一层就有(1000*1000+1)*100=1亿参数，这实在是太多了！我们看到图像只扩大一点，参数数量就会多很多，因此它的扩展性很差。\n",
    "\n",
    "    没有利用像素之间的位置信息 \n",
    "\n",
    "对于图像识别任务来说，每个像素和其周围像素的联系是比较紧密的，和离得很远的像素的联系可能就很小了。如果一个神经元和上一层所有神经元相连，那么就相当于对于一个像素来说，把图像的所有像素都等同看待，这不符合前面的假设。当我们完成每个连接权重的学习之后，最终可能会发现，有大量的权重，它们的值都是很小的。努力学习大量并不重要的权重，这样的学习必将是非常低效的。\n",
    "\n",
    "    网络层数限制\n",
    "\n",
    " 我们知道网络层数越多其表达能力越强，但是通过梯度下降方法训练深度全连接神经网络很困难，因为全连接神经网络的梯度很难传递超过3层。因此，我们不可能得到一个很深的全连接神经网络，也就限制了它的能力。\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 卷积神经网络：\n",
    "适合做图像识别：\n",
    "\n",
    "    局部连接 \n",
    "\n",
    "每个神经元不再和上一层的所有神经元相连，而只和一小部分神经元相连。这样就减少了很多参数。\n",
    "\n",
    "    权值共享 \n",
    "\n",
    "一组连接可以共享同一个权重，而不是每个连接有一个不同的权重，这样又减少了很多参数。\n",
    "\n",
    "    下采样 \n",
    "\n",
    "可以使用Pooling来减少每层的样本数，进一步减少参数数量，同时还可以提升模型的鲁棒性。\n",
    "\n",
    "\n",
    "对于图像识别任务来说，卷积神经网络通过尽可能保留重要的参数，去掉大量不重要的参数，来达到更好的学习效果。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 所以这里的CNN效果比全链接的好"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "varInspector": {
   "cols": {
    "lenName": 16,
    "lenType": 16,
    "lenVar": 40
   },
   "kernels_config": {
    "python": {
     "delete_cmd_postfix": "",
     "delete_cmd_prefix": "del ",
     "library": "var_list.py",
     "varRefreshCmd": "print(var_dic_list())"
    },
    "r": {
     "delete_cmd_postfix": ") ",
     "delete_cmd_prefix": "rm(",
     "library": "var_list.r",
     "varRefreshCmd": "cat(var_dic_list()) "
    }
   },
   "types_to_exclude": [
    "module",
    "function",
    "builtin_function_or_method",
    "instance",
    "_Feature"
   ],
   "window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
